[issue23174] shelve.open fails with error anydbm.error: db type could not be determined
Karl Richter added the comment: That's nice, thanks. Considering your last comment, some points * If the issue can't go into the error message, than the essence of the discussion here should go into the docs (in 0.5 to 1.5 sentences). * It'd be nice if it was clear from the error message that shelve or underlying modules check the extension in filename, i.e. do `anydbm.error: db type could not be determined by file extension '%s' in filename '%s'` rather than `anydbm.error: db type could not be determined`. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23174 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23174] shelve.open fails with error anydbm.error: db type could not be determined
Karl Richter added the comment: Then, let the error message say You are opening a just-created empty file. The db type of the file cannot, therefore, be determined. which is much clearer than anydbm.error: db type could not be determined which sounds like a generic fallback error message in an error occured-style. It seems to be necessary that the filename passed to `shelve.open` has a suffix which is not very intuitive for Linux users. It'd be great to have validation of the suffix and raise a `ValueError` with an error message like `filename '%s' doesn't contain a suffix % (filename,)` when it isn't supplied. If there's another issue regarding the fact that the file is just-created and empty, this is independent of the issue above. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23174 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23174] shelve.open fails with error anydbm.error: db type could not be determined
New submission from Karl Richter: `shelve.open(tempfile.mkstemp()[1])` fails with error anydbm.error: db type could not be determined which is not explainable with the docs. Traceback is Traceback (most recent call last): File ./cudaminer_param_checker.py, line 720, in module plac.call(cudaminer_param_checker) File /usr/local/lib/python2.7/dist-packages/plac_core.py, line 309, in call cmd, result = parser_from(obj).consume(arglist) File /usr/local/lib/python2.7/dist-packages/plac_core.py, line 195, in consume return cmd, self.func(*(args + varargs + extraopts), **kwargs) File ./cudaminer_param_checker.py, line 715, in cudaminer_param_checker visualize_cudaminer_param_checker_results_wxpython_gui() File ./cudaminer_param_checker.py, line 365, in visualize_cudaminer_param_checker_results_wxpython_gui frame = CudaminerParamChecker(None, ) File ./cudaminer_param_checker.py, line 378, in __init__ self.generator = CudaminerParamCheckerGenerator() File ./cudaminer_param_checker.py, line 160, in __init__ self.result_dict= shelve.open(storage_file_path) File /usr/lib/python2.7/shelve.py, line 239, in open return DbfilenameShelf(filename, flag, protocol, writeback) File /usr/lib/python2.7/shelve.py, line 223, in __init__ Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback) File /usr/lib/python2.7/anydbm.py, line 82, in open raise error, db type could not be determined anydbm.error: db type could not be determined -- assignee: docs@python components: Documentation messages: 233488 nosy: docs@python, krichter priority: normal severity: normal status: open title: shelve.open fails with error anydbm.error: db type could not be determined type: behavior versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23174 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23174] shelve.open fails with error anydbm.error: db type could not be determined
Karl Richter added the comment: EDIT 1: other examples, e.g. import os import shelve curdir = os.path.dirname(__file__) passwords = shelve.open(os.path.join(curdir, 'password_db')) work, so there's need for usable error messages. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23174 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23163] pdb docs need to contain a statement on threads/multithreaded debugging
New submission from Karl Richter: Due to the fact that `pdb` currently simply ignores breakpoints which are set and hit in another than the main thread the docs need to contain a statement on behavior in a multithreaded environment. -- components: Library (Lib) messages: 233409 nosy: krichter priority: normal severity: normal status: open title: pdb docs need to contain a statement on threads/multithreaded debugging type: enhancement versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23163 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18119] urllib.FancyURLopener does not treat URL fragments correctly
karl added the comment: OK I fixed the code. The issue is here https://hg.python.org/cpython/file/1e1c6e306eb4/Lib/urllib/request.py#l656 newurl = urlunparse(urlparts) Basically it reinjects the fragment in the new url. The fix is easy. if urlparts.fragment: urlparts = list(urlparts) urlparts[5] = newurl = urlunparse(urlparts) I was trying to make a test for it, but failed. Could someone help me for the test so I can complete the patch? Added the code patch only. -- keywords: +patch Added file: http://bugs.python.org/file36832/issue18119-code-only.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18119 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18119] urllib.FancyURLopener does not treat URL fragments correctly
karl added the comment: This is the correct behavior GET http://example.com/foo with a response containing 302 and Location: /bar#test must trigger http://example.com/bar#test Location is defined in http://tools.ietf.org/html/rfc7231#section-7.1.2 7.1.2. Location The Location header field is used in some responses to refer to a specific resource in relation to the response. The type of relationship is defined by the combination of request method and status code semantics. Location = URI-reference The field value consists of a single URI-reference. When it has the form of a relative reference ([RFC3986], Section 4.2), the final value is computed by resolving it against the effective request URI ([RFC3986], Section 5). A bit after in the spec. If the Location value provided in a 3xx (Redirection) response does not have a fragment component, a user agent MUST process the redirection as if the value inherits the fragment component of the URI reference used to generate the request target (i.e., the redirection inherits the original reference's fragment, if any). For example, a GET request generated for the URI reference http://www.example.org/~tim; might result in a 303 (See Other) response containing the header field: Location: /People.html#tim which suggests that the user agent redirect to http://www.example.org/People.html#tim; Likewise, a GET request generated for the URI reference http://www.example.org/index.html#larry; might result in a 301 (Moved Permanently) response containing the header field: Location: http://www.example.net/index.html which suggests that the user agent redirect to http://www.example.net/index.html#larry;, preserving the original fragment identifier. -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18119 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18119] urllib.FancyURLopener does not treat URL fragments correctly
karl added the comment: Takahashi-san, Ah sorry misunderstood which part your were talking about. I assume wrongly you were talking about navigation. Yes for the request which is sent to the server it should be http://tools.ietf.org/html/rfc7230#section-5.3.1 So refactoring your example. 1st request: GET /foo HTTP/1.1 Accept: text/html Host: example.com server response HTTP/1.1 302 Found Location: /bar#test second request must be GET /bar HTTP/1.1 Accept: text/html Host: example.com As for the navigation context is indeed part of the piece of code taking in charge the document after being parsed and not the one doing the HTTP request. (putting it here just that people understand) (to be tested) For server side receiving invalid Request-line http://tools.ietf.org/html/rfc7230#section-3.1.1 Recipients of an invalid request-line SHOULD respond with either a 400 (Bad Request) error or a 301 (Moved Permanently) redirect with the request-target properly encoded. A recipient SHOULD NOT attempt to autocorrect and then process the request without a redirect, since the invalid request-line might be deliberately crafted to bypass security filters along the request chain. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18119 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18119] urllib.FancyURLopener does not treat URL fragments correctly
karl added the comment: In class urlopen_HttpTests https://hg.python.org/cpython/file/4f314dedb84f/Lib/test/test_urllib.py#l191 there is a test for invalid redirects def test_invalid_redirect(self): https://hg.python.org/cpython/file/4f314dedb84f/Lib/test/test_urllib.py#l247 And one for fragments def test_url_fragment(self): https://hg.python.org/cpython/file/4f314dedb84f/Lib/test/test_urllib.py#l205 which refers to http://bugs.python.org/issue11703 code in https://hg.python.org/cpython/file/d5688a94a56c/Lib/urllib.py -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18119 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment: Ok my tests are ok. → ./python.exe -m unittest -v Lib/test/test_urllib2net.py test_close (Lib.test.test_urllib2net.CloseSocketTest) ... ok test_custom_headers (Lib.test.test_urllib2net.OtherNetworkTests) ... FAIL test_file (Lib.test.test_urllib2net.OtherNetworkTests) ... test_ftp (Lib.test.test_urllib2net.OtherNetworkTests) ... skipped Resource 'ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/00README-Legal-Rules-Regs' is not available test_headers_case_sensitivity (Lib.test.test_urllib2net.OtherNetworkTests) ... ok test_redirect_url_withfrag (Lib.test.test_urllib2net.OtherNetworkTests) ... ok test_sites_no_connection_close (Lib.test.test_urllib2net.OtherNetworkTests) ... ok test_urlwithfrag (Lib.test.test_urllib2net.OtherNetworkTests) ... ok test_ftp_basic (Lib.test.test_urllib2net.TimeoutTest) ... ok test_ftp_default_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok test_ftp_no_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok test_ftp_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok test_http_basic (Lib.test.test_urllib2net.TimeoutTest) ... ok test_http_default_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok test_http_no_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok test_http_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok I wonder if test_custom_headers fails because of my modifications. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22478] tests for urllib2net are in bad shapes
New submission from karl:
→ ./python.exe -V
Python 3.4.2rc1+
→ hg tip
changeset: 92532:6dcc96fa3970
tag: tip
parent: 92530:ad45c2707006
parent: 92531:8eb4eec8626c
user:Benjamin Peterson benja...@python.org
date:Mon Sep 22 22:44:21 2014 -0400
summary: merge 3.4 (#22459)
When working on issue #5550, I realized that some tests are currently failing.
Here the log of running:
→ ./python.exe -m unittest -v Lib/test/test_urllib2net.py
test_close (Lib.test.test_urllib2net.CloseSocketTest) ... ok
test_custom_headers (Lib.test.test_urllib2net.OtherNetworkTests) ... FAIL
test_file (Lib.test.test_urllib2net.OtherNetworkTests) ... test_ftp
(Lib.test.test_urllib2net.OtherNetworkTests) ... skipped Resource
'ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/00README-Legal-Rules-Regs'
is not available
test_redirect_url_withfrag (Lib.test.test_urllib2net.OtherNetworkTests) ... ok
test_sites_no_connection_close (Lib.test.test_urllib2net.OtherNetworkTests) ...
ok
test_urlwithfrag (Lib.test.test_urllib2net.OtherNetworkTests) ... ok
test_ftp_basic (Lib.test.test_urllib2net.TimeoutTest) ... ok
test_ftp_default_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok
test_ftp_no_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok
test_ftp_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok
test_http_basic (Lib.test.test_urllib2net.TimeoutTest) ... ok
test_http_default_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok
test_http_no_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok
test_http_timeout (Lib.test.test_urllib2net.TimeoutTest) ... ok
==
ERROR: test_file (Lib.test.test_urllib2net.OtherNetworkTests)
(url='file:/Users/karl/code/cpython/%40test_61795_tmp')
--
Traceback (most recent call last):
File /Users/karl/code/cpython/Lib/test/test_urllib2net.py, line 243, in
_test_urls
f = urlopen(url, req, TIMEOUT)
File /Users/karl/code/cpython/Lib/test/test_urllib2net.py, line 33, in
wrapped
return _retry_thrice(func, exc, *args, **kwargs)
File /Users/karl/code/cpython/Lib/test/test_urllib2net.py, line 23, in
_retry_thrice
return func(*args, **kwargs)
File /Users/karl/code/cpython/Lib/urllib/request.py, line 447, in open
req = Request(fullurl, data)
File /Users/karl/code/cpython/Lib/urllib/request.py, line 267, in __init__
origin_req_host = request_host(self)
File /Users/karl/code/cpython/Lib/urllib/request.py, line 250, in
request_host
host = _cut_port_re.sub(, host, 1)
TypeError: expected string or buffer
==
ERROR: test_file (Lib.test.test_urllib2net.OtherNetworkTests)
(url=('file:///nonsensename/etc/passwd', None, class 'urllib.error.URLError'))
--
Traceback (most recent call last):
File /Users/karl/code/cpython/Lib/test/test_urllib2net.py, line 243, in
_test_urls
f = urlopen(url, req, TIMEOUT)
File /Users/karl/code/cpython/Lib/test/test_urllib2net.py, line 33, in
wrapped
return _retry_thrice(func, exc, *args, **kwargs)
File /Users/karl/code/cpython/Lib/test/test_urllib2net.py, line 23, in
_retry_thrice
return func(*args, **kwargs)
File /Users/karl/code/cpython/Lib/urllib/request.py, line 447, in open
req = Request(fullurl, data)
File /Users/karl/code/cpython/Lib/urllib/request.py, line 267, in __init__
origin_req_host = request_host(self)
File /Users/karl/code/cpython/Lib/urllib/request.py, line 250, in
request_host
host = _cut_port_re.sub(, host, 1)
TypeError: expected string or buffer
==
FAIL: test_custom_headers (Lib.test.test_urllib2net.OtherNetworkTests)
--
Traceback (most recent call last):
File /Users/karl/code/cpython/Lib/test/test_urllib2net.py, line 186, in
test_custom_headers
self.assertEqual(request.get_header('User-agent'), 'Test-Agent')
AssertionError: 'Python-urllib/3.4' != 'Test-Agent'
- Python-urllib/3.4
+ Test-Agent
--
Ran 16 tests in 124.879s
FAILED (failures=1, errors=2, skipped=1)
--
components: Tests
messages: 227417
nosy: karlcow
priority: normal
severity: normal
status: open
title: tests for urllib2net are in bad shapes
versions: Python 3.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22478
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment: Opened issue #22478 for the tests failing. Not related to my modification. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22478] tests for urllib2net are in bad shapes
karl added the comment:
ok let's see
→ ./python.exe -m unittest -v
Lib.test.test_urllib2net.OtherNetworkTests.test_custom_headers
test_custom_headers (Lib.test.test_urllib2net.OtherNetworkTests) ... FAIL
==
FAIL: test_custom_headers (Lib.test.test_urllib2net.OtherNetworkTests)
--
Traceback (most recent call last):
File /Users/karl/code/cpython/Lib/test/test_urllib2net.py, line 186, in
test_custom_headers
self.assertEqual(request.get_header('User-agent'), 'Test-Agent')
AssertionError: 'Python-urllib/3.4' != 'Test-Agent'
- Python-urllib/3.4
+ Test-Agent
--
Ran 1 test in 0.551s
FAILED (failures=1)
→ ./python.exe
Python 3.4.2rc1+ (3.4:8eb4eec8626c+, Sep 23 2014, 21:53:11)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.51)] on darwin
Type help, copyright, credits or license for more information.
import urllib.request
url = 'http://127.0.0.1/'
opener = urllib.request.build_opener()
request = urllib.request.Request(url)
request.header_items()
[]
request.headers
{}
request.add_header('User-Agent', 'Test-Agent')
request.headers
{'User-agent': 'Test-Agent'}
request.header_items()
[('User-agent', 'Test-Agent')]
opener.open(request)
http.client.HTTPResponse object at 0x10c0aedc0
request.get_header('User-agent'), 'Test-Agent'
('Test-Agent', 'Test-Agent')
request.header_items()
[('User-agent', 'Test-Agent'), ('Host', '127.0.0.1')]
request.headers
{'User-agent': 'Test-Agent'}
OK so far so good.
And my server recorded
127.0.0.1 - - [24/Sep/2014:17:07:41 +0900] GET / HTTP/1.1 200 9897 -
Test-Agent
Let's do it the way, the test has been designed.
→ ./python.exe
Python 3.4.2rc1+ (3.4:8eb4eec8626c+, Sep 23 2014, 21:53:11)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.51)] on darwin
Type help, copyright, credits or license for more information.
import urllib.request
url = 'http://127.0.0.1/'
opener = urllib.request.build_opener()
request = urllib.request.Request(url)
request.header_items()
[]
opener.open(request)
http.client.HTTPResponse object at 0x10e05aa80
request.header_items()
[('User-agent', 'Python-urllib/3.4'), ('Host', '127.0.0.1')]
request.has_header('User-agent')
True
request.add_header('User-Agent', 'Test-Agent')
opener.open(request)
http.client.HTTPResponse object at 0x10e05ab50
request.get_header('User-agent'), 'Test-Agent'
('Python-urllib/3.4', 'Test-Agent')
request.add_header('Foo', 'bar')
request.header_items()
[('User-agent', 'Test-Agent'), ('Host', '127.0.0.1'), ('Foo', 'bar')]
opener.open(request)
http.client.HTTPResponse object at 0x10e05ad58
request.header_items()
[('User-agent', 'Test-Agent'), ('Host', '127.0.0.1'), ('Foo', 'bar')]
request.get_header('User-agent'), 'Test-Agent'
('Python-urllib/3.4', 'Test-Agent')
request.headers
{'User-agent': 'Test-Agent', 'Foo': 'bar'}
And the server recorded.
127.0.0.1 - - [24/Sep/2014:17:12:52 +0900] GET / HTTP/1.1 200 9897 -
Python-urllib/3.4
127.0.0.1 - - [24/Sep/2014:17:12:52 +0900] GET / HTTP/1.1 200 9897 -
Python-urllib/3.4
127.0.0.1 - - [24/Sep/2014:17:14:15 +0900] GET / HTTP/1.1 200 9897 -
Python-urllib/3.4
So it seems that User-Agent is immutable once it has been set the first time.
Not in the same dictionary.
request.unredirected_hdrs
{'User-agent': 'Python-urllib/3.4', 'Host': '127.0.0.1'}
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22478
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22478] tests for urllib2net are in bad shapes
karl added the comment:
Ah! the User-Agent (or anything which is in unredirected_hdrs) will not be
updated if it has already been set once.
https://hg.python.org/cpython/file/064f6baeb6bd/Lib/urllib/request.py#l1154
headers = dict(request.unredirected_hdrs)
headers
{'User-agent': 'Python-urllib/3.4', 'Host': '127.0.0.1'}
request.headers
{'User-agent': 'Test-Agent', 'Foo': 'cool'}
headers.update(dict((k, v) for k, v in request.headers.items() if k not in
headers))
headers
{'User-agent': 'Python-urllib/3.4', 'Host': '127.0.0.1', 'Foo': 'cool'}
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22478
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment: OK after fixing my repo (Thanks orsenthil) I got the tests running properly. The inspection order of the two dictionary was not right, so I had to modify a bit the patch. → ./python.exe -m unittest -v Lib.test.test_urllib2net.OtherNetworkTests.test_headers_case_sensitivity test_headers_case_sensitivity (Lib.test.test_urllib2net.OtherNetworkTests) ... ok -- Ran 1 test in 0.286s OK → ./python.exe -m unittest -v Lib.test.test_urllib2net.OtherNetworkTests.test_custom_headers test_custom_headers (Lib.test.test_urllib2net.OtherNetworkTests) ... ok -- Ran 1 test in 0.575s OK New patch issue5550-5.patch unlinking issue5550-4.patch -- Added file: http://bugs.python.org/file36717/issue5550-5.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
Changes by karl karl+pythonb...@la-grange.net: Removed file: http://bugs.python.org/file36698/issue-5550-4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15799] httplib client and statusline
karl added the comment:
Let's close this.
HTTP/1.1301 .split(None, 2)
['HTTP/1.1', '301']
HTTP/1.1301 .split(' ', 2)
['HTTP/1.1', '', ' 301 ']
I think it would be nice to have a way to warn without stopping, but the last
comment from r.david.murray makes sense too. :)
--
resolution: - not a bug
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15799
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17319] http.server.BaseHTTPRequestHandler send_response_only doesn't check the type and value of the code.
karl added the comment: Where this is defined in the new RFC. http://tools.ietf.org/html/rfc7230#section-3.1.2 status-line = HTTP-version SP status-code SP reason-phrase CRLF Things to enforce status-code= 3DIGIT Response status code are now defined in http://tools.ietf.org/html/rfc7231#section-6 with something important. HTTP status codes are extensible. HTTP clients are not required to understand the meaning of all registered status codes, though such understanding is obviously desirable. However, a client MUST understand the class of any status code, as indicated by the first digit, and treat an unrecognized status code as being equivalent to the x00 status code of that class, with the exception that a recipient MUST NOT cache a response with an unrecognized status code. For example, if an unrecognized status code of 471 is received by a client, the client can assume that there was something wrong with its request and treat the response as if it had received a 400 (Bad Request) status code. The response message will usually contain a representation that explains the status. That should help. The full registry of status code is defined here http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml @dmi.baranov In the patch +def _is_valid_status_code(code): +return isinstance(code, int) and 0 = code = 999 Maybe there is a missing check where the len(str(code)) == 3 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17319 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment: Ok this is an attempt at solving the issue with lowercase. I find my get_header a bit complicated, but if you have a better idea. :) I'll modify the patches. I have try to run the tests on the mac here but I have an issue currently. → ./python.exe -V Python 3.5.0a0 Traceback (most recent call last): File ./Tools/scripts/patchcheck.py, line 6, in module import subprocess File /Users/karl/code/cpython/Lib/subprocess.py, line 353, in module import signal ImportError: No module named 'signal' make: *** [patchcheck] Error 1 -- Added file: http://bugs.python.org/file36695/issue-5550-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
Changes by karl karl+pythonb...@la-grange.net: Removed file: http://bugs.python.org/file36695/issue-5550-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment: And I had to do a typo in patch3. Submitting patch4. Sorry about that. -- Added file: http://bugs.python.org/file36698/issue-5550-4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces (and other weird stuff)
karl added the comment: Just a follow up for giving the stable version of the now new RFC version for HTTP 1.1 HTTP header field names parsing http://tools.ietf.org/html/rfc7230#section-3.2.4 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17322 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21885] shutil.copytree hangs (on copying root directory of a lxc container) (should succeed or raise exception nested)
New submission from Karl Richter: reproduction (on Ubuntu 14.04 amd64 with lxc 1.0.4) (with python 2.7.6 and 3.4.0) # as root/with privileges lxc-create -n ubuntu-trusty-amd64 -t ubuntu -- --arch amd64 --release trusty lxc-stop -n ubuntu-trusty-amd64 # assert container isn't running cd /var/lib/lxc python import shutil shutil.copytree(ubuntu-trusty-amd64, ubuntu-trusty-amd64-orig) # never returns (after a multiple of the time rsync needs (see below) no more I/O operations) verify behavior of rsync (3.1.0): # as root/with privileges rsync -a ubuntu-trusty-amd64/ ubuntu-trusty-amd64-orig/ # succeeds If the container is shutdown it should no longer point to system resources, and thus be able to get stuck on reading from a device file (and should rsync get stuck as well in this case?). It would be nice if python fails with an exception (or succeeds, of course) instead of getting stuck. -- messages: 221960 nosy: krichter priority: normal severity: normal status: open title: shutil.copytree hangs (on copying root directory of a lxc container) (should succeed or raise exception nested) versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21885 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21889] https://docs.python.org/2/library/multiprocessing.html#process-and-exceptions doesn't explain exception
New submission from Karl Richter: Although the section https://docs.python.org/2/library/multiprocessing.html#process-and-exceptions (of the multiprocessing module documentation) is titled ... and exceptions it doesn't say anything about exceptions. I assume that it behaves like the thread API (as stated and referenced in the introduction of the module doc). This implies though that either the reference is limited to that statement (- remove and exceptions from the header as there's no special section on them because everything can be found in thread API) or add an explicit reference to the thread API. If this assumption is wrong the section is badly organized or doesn't make any sense at all. I'm not yet sure about exception handling in the multiprocessing module in case it's different from threads, but that shouldn't matter for this doc issue report. I'd also like to suggest a more detailed section on exceptions with usage of queues to pass them as objects to the parent or another process. -- assignee: docs@python components: Documentation messages: 221987 nosy: docs@python, krichter priority: normal severity: normal status: open title: https://docs.python.org/2/library/multiprocessing.html#process-and-exceptions doesn't explain exception type: enhancement versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21889 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] datetime: add ability to parse RFC 3339 dates and times
karl added the comment: After inspections, the best library for parsing RFC3339 style date is definitely: https://github.com/tonyg/python-rfc3339/ Main code at https://github.com/tonyg/python-rfc3339/blob/master/rfc3339.py -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15873 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] datetime: add ability to parse RFC 3339 dates and times
karl added the comment: I had the issue today. I needed to parse a date with the following format. 2014-04-04T23:59:00+09:00 and could not with strptime. I see a discussion in March 2014 http://code.activestate.com/lists/python-ideas/26883/ but no followup. For references: http://www.w3.org/TR/NOTE-datetime http://tools.ietf.org/html/rfc3339 -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15873 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] datetime: add ability to parse RFC 3339 dates and times
karl added the comment: On closer inspection, Anders Hovmöller proposal doesn't work. https://github.com/boxed/iso8601 At least for the microseconds part. In http://tools.ietf.org/html/rfc3339#section-5.6, the microsecond part is defined as: time-secfrac= . 1*DIGIT In http://www.w3.org/TR/NOTE-datetime, same thing: s= one or more digits representing a decimal fraction of a second Anders considers it to be only six digits. It can be more or it can be less. :) Will comment on github too. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15873 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] datetime: add ability to parse RFC 3339 dates and times
karl added the comment: Noticed some people doing the same thing https://github.com/tonyg/python-rfc3339 http://home.blarg.net/~steveha/pyfeed.html https://wiki.python.org/moin/WorkingWithTime -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15873 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12455] urllib2 forces title() on header names, breaking some requests
karl added the comment: Mark, I'm happy to followup. I will be in favor of removing any capitalization and not to change headers whatever they are. Because it doesn't matter per spec. Browsers do not care about the capitalization. And I haven't identified Web Compatibility issues regarding the capitalization. That said, it seems that Cal msg139512 had an issue, I would love to know which server/API had this behavior to fill a but at http://webcompat.com/ So… Where do we stand? Feature or removing anything which modifies the capitalization of headers? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12455 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment: @Mark, yup, I can do that. I just realized that since my contribution there was a PSF Contributor Agreement. This is signed. I need to dive a bit again in the code to remember where things fail. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
karl added the comment: Mark, The code is using urllib for demonstrating the issue with wikipedia and other sites which are blocking python-urllib user agents because it is used by many spam harvesters. The proposal is about giving a possibility in robotparser lib to add a feature for setting up the user-agent. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15851 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
karl added the comment: Note that one of the proposal is to just document in https://docs.python.org/3/library/urllib.robotparser.html the proposal made in msg169722 (available in 3.4+) robotparser.URLopener.version = 'MyVersion' -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15851 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
karl added the comment:
→ python
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type help, copyright, credits or license for more information.
import robotparser
rp = robotparser.RobotFileParser('http://somesite.test.site/robots.txt')
rp.read()
Let's check the server logs:
127.0.0.1 - - [23/Jun/2014:08:44:37 +0900] GET /robots.txt HTTP/1.0 200 92
- Python-urllib/1.17
Robotparser by default was using in 2.* the Python-urllib/1.17 user agent which
is traditionally blocked by many sysadmins. A solution has been already
proposed above:
This is the proposed test for 3.4
import urllib.robotparser
import urllib.request
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'MyUa/0.1')]
urllib.request.install_opener(opener)
rp = urllib.robotparser.RobotFileParser('http://localhost:')
rp.read()
The issue is not anymore about changing the lib, but just about documenting on
how to change the RobotFileParser default UA. We can change the title of this
issue if it's confusing. Or close it and open a new one for documenting what
makes it easier :)
Currently robotparser.py imports urllib user agent.
http://hg.python.org/cpython/file/7dc94337ef67/Lib/urllib/request.py#l364
It's a common failure we encounter when using urllib in general, including
robotparser.
As for wikipedia, they fixed their server side user agent sniffing, and do not
filter anymore python-urllib.
GET /robots.txt HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, compress
Host: en.wikipedia.org
User-Agent: Python-urllib/1.17
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 3161
Cache-control: s-maxage=3600, must-revalidate, max-age=0
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 5208
Content-Type: text/plain; charset=utf-8
Date: Sun, 22 Jun 2014 23:59:16 GMT
Last-modified: Tue, 26 Nov 2013 17:39:43 GMT
Server: Apache
Set-Cookie: GeoIP=JP:Tokyo:35.6850:139.7514:v4; Path=/; Domain=.wikipedia.org
Vary: X-Subdomain
Via: 1.1 varnish, 1.1 varnish, 1.1 varnish
X-Article-ID: 19292575
X-Cache: cp1065 miss (0), cp4016 hit (1), cp4009 frontend hit (215)
X-Content-Type-Options: nosniff
X-Language: en
X-Site: wikipedia
X-Varnish: 2529666795, 2948866481 2948865637, 4134826198 4130750894
Many other sites still do. :)
--
versions: +Python 3.4 -Python 3.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15851
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21739] Add hint about expression in list comprehensions (https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions)
New submission from Karl Richter: It would be useful to have a short statement in the docs (https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions) that the expression in a list comprehension isn't put into a block, but evaluated against the same block where it is located because intuitively one might think that variables can be overridden in the statement. This applies to both 2.7 and 3.4.1. -- assignee: docs@python components: Documentation messages: 220374 nosy: docs@python, krichter priority: normal severity: normal status: open title: Add hint about expression in list comprehensions (https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions) type: enhancement versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21739 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21208] Change default behavior of arguments with type bool when options are specified
Karl Richter added the comment: @paul.j3 That's interesting [1]. Documenting argparse.register seems crucial to me (- reopen or file a new request?). After dealing more with the very sophisticated and complex functionality of argparse I'm sure that this is the only use case where such an unintuitive result may happen, so it's worth adding an explicit negative example in the docs of argparse.add_argument with a hint to the docs about way that argparse handles types. -- [1] (and a very good example for missing capabilities of QA sites with regard to the relation of votes of your much better answer and the one with the highest number of votes) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21208 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21325] Missing Generic EXIF library for images in the standard library
New submission from karl: There is a room for a consistent and good EXIF library for the Python Standard Library. -- components: Library (Lib) messages: 216978 nosy: karlcow priority: normal severity: normal status: open title: Missing Generic EXIF library for images in the standard library type: enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21325 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21208] Change default behavior of arguments with type bool when options are specified
Karl Richter added the comment: That's a pity, I still think it's confusing. Thanks for your feedback! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21208 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21208] Change default behavior of arguments with type bool when options are specified
New submission from Karl Richter: As arguments with type bool are the only ones whose values can be manipulated without passing an option to the argument on CLI, the default behavior for those should be changed from ignoring options to failing when options are specified. Consider the following example: code import argparse cache_size_default=1000 cache_size_option = c cache_size_option_long = cache-size start_db_default = False start_db_option = t start_db_option_long = start-db parser = argparse.ArgumentParser(description='Process some integers.') parser.add_argument(-%s % start_db_option, --%s % start_db_option_long, default=start_db_default, type=bool, nargs='?', help='@TODO', dest=start_db_option_long) parser.add_argument(-%s % cache_size_option, --%s % cache_size_option_long, type=int, nargs='?', default=cache_size_default, help='size of osm2pgsql cache', dest=cache_size_option_long) def osm_postgis_transform(cache_size=cache_size_default, start_db=start_db_default): print(start_db, cache_size) if __name__ == __main__: args = vars(parser.parse_args()) osm_postgis_transform(cache_size=args[cache_size_option_long], start_db=args[start_db_option_long]) /code When the script is invoked with code python issue-argparse.py --start-db False --cache-size 2000 /code The value for start-db is True which is not really intuitive. Changing the default behavior to either fail at argument parsing or overwrite the argument value would be an improvement in my opinion. -- components: Library (Lib) messages: 215983 nosy: krichter priority: normal severity: normal status: open title: Change default behavior of arguments with type bool when options are specified type: behavior versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21208 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21208] Change default behavior of arguments with type bool when options are specified
Karl Richter added the comment: I've been mistaken about the behavior if no argument is specified (then the argument value is None), so this is a bug! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21208 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20683] [doc] Ease comprehension of section 9.2 of docs for Python 2 and 3 with example
New submission from Karl Richter: The explanation of namespaces in section 9.2 in documentation (http://docs.python.org/3/tutorial/classes.html#python-scopes-and-namespaces) is just so complicated without (at least one tiny) example. The example would ease the comprehension of the section by a high factor. -- assignee: docs@python components: Documentation messages: 211607 nosy: docs@python, krichter priority: normal severity: normal status: open title: [doc] Ease comprehension of section 9.2 of docs for Python 2 and 3 with example type: enhancement versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20683 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue747320] rfc2822 formatdate functionality duplication
karl added the comment: Eric, what do you recommend to move forward with this bug and patches? Need guidance. Do you have an example for (A minor thing: I would use “attribute” instead of “variable” in the docstrings.) Also which code base I should use? A lot of water has gone under the bridge in one year. :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue747320 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11448] docs for HTTPConnection.set_tunnel are ambiguous
karl added the comment: ooops right, my bad. s/on port 8080. We first/on port 8080, we first/ better? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11448 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19084] No way to use TLS-PSK from python ssl
New submission from Karl Palsson: OpenSSL supports TLS-PSK which some people (myself obviously) find to be substantially easier to use than setting up certs. However, there's no way to use PSK via the current SSL api in python. It would be very nice to be able to use PSK from python. For OpenSSL, even the C API is particularly easy. Attached is the implementation used in Mosquitto, a MQTT message broker that supports both cert based and PSK based TLS. -- components: Library (Lib) files: 5bcfpEKD.txt messages: 198362 nosy: karlp priority: normal severity: normal status: open title: No way to use TLS-PSK from python ssl type: behavior versions: Python 2.7, Python 3.4 Added file: http://bugs.python.org/file31859/5bcfpEKD.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19084 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment: Yes case insensitive make sense. It seems it will require a lot of modifications in the code. So I guess maybe the first step is to identify where it could break. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment: orsenthil, Added test for python 3.3 for testing the case insensitivity for has_header and get_header. The tests fail as it should. See issue-5550-test-1.patch I will make a patch to urllib/request.py to make it work. OK with you? -- keywords: +patch Added file: http://bugs.python.org/file29505/issue-5550-test-1.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5550 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment:
First, Sanity check for myself to be sure to understand.
==
→ python3.3
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type help, copyright, credits or license for more information.
import urllib.request
req = urllib.request.Request('http://www.python.org/')
req.headers
{}
req.unredirected_hdrs
{}
r = urllib.request.urlopen(req)
req.headers
{}
req.unredirected_hdrs
{'User-agent': 'Python-urllib/3.3', 'Host': 'www.python.org'}
req.header_items()
[('User-agent', 'Python-urllib/3.3'), ('Host', 'www.python.org')]
req.has_header('host')
False
req.has_header('Host')
True
req.get_header('host')
req.get_header('Host')
'www.python.org'
'host' in req.unredirected_hdrs
False
'Host' in req.unredirected_hdrs
True
'host' in req.header_items()
False
'Host' in req.header_items()
False
req.unredirected_hdrs.items()
dict_items([('User-agent', 'Python-urllib/3.3'), ('Host', 'www.python.org')])
req.headers.get('Host', req.unredirected_hdrs.get('Host',None))
'www.python.org'
req.headers.get('Host')
req.headers.get('host'.capitalize(),
req.unredirected_hdrs.get('host'.capitalize(),None))
'www.python.org'
req.headers.get('HOST'.capitalize(),
req.unredirected_hdrs.get('HOST'.capitalize(),None))
'www.python.org'
req.headers.get('host'.title(),
req.unredirected_hdrs.get('host'.title(),None))
'www.python.org'
'host'.capitalize() in req.unredirected_hdrs
True
==
OK. The two add methods force the capitalization thought capitalize() (And not
title() which is an issue by itself)
http://hg.python.org/cpython/file/3.3/Lib/urllib/request.py#l359
def add_header(self, key, val):
# useful for something like authentication
self.headers[key.capitalize()] = val
def add_unredirected_header(self, key, val):
# will not be added to a redirected request
self.unredirected_hdrs[key.capitalize()] = val
HTTP headers are case insensitive. The way the methods get_header() and
has_header() are currently designed. We could also capitalize the variable
before requesting it.
So something like
def get_header(self, header_name, default=None):
return self.headers.get(
header_name.capitalize(),
self.unredirected_hdrs.get(header_name.capitalize(), default))
def has_header(self, header_name):
return (header_name.capitalize() in self.headers or
header_name.capitalize() in self.unredirected_hdrs)
The method to add headers on request is
req.add_header(foo-bar,booh)
req.headers
{'Foo-bar': 'booh'}
req.unredirected_hdrs
{'User-agent': 'Python-urllib/3.3', 'Host': 'www.python.org'}
req.header_items()
[('Foo-bar', 'booh'), ('User-agent', 'Python-urllib/3.3'), ('Host',
'www.python.org')]
So if someone add an header it will be capitalized. And the query will be
correct.
The issue is more with addheader which doesn't have the same constraint.
http://hg.python.org/cpython/file/3.3/Lib/urllib/request.py#l1624
Personally I would have made everything case insensitive ;)
Also note that in this module the casing is not consistent when the values are
hardcoded. Sometimes Content-Type, sometimes Content-type.
Anyway I submitted a patch with the code modification AND the test. And this is
the result when running the test suite.
→ ./python.exe Lib/test/test_urllib2net.py
test_sni (__main__.HTTPSTests) ... skipped 'test disabled - test server needed'
test_custom_headers (__main__.OtherNetworkTests) ... ok
test_file (__main__.OtherNetworkTests) ... ok
test_ftp (__main__.OtherNetworkTests) ... ok
test_headers_case_sensitivity (__main__.OtherNetworkTests) ... ok
test_sites_no_connection_close (__main__.OtherNetworkTests) ...
/Users/karl/Documents/2011/cpython/Lib/socket.py:370: ResourceWarning: unclosed
socket.socket object, fd=5, family=2, type=1, proto=6
self._sock = None
ok
test_urlwithfrag (__main__.OtherNetworkTests) ... ok
test_close (__main__.CloseSocketTest) ... ok
test_ftp_basic (__main__.TimeoutTest) ... ok
test_ftp_default_timeout (__main__.TimeoutTest) ... ok
test_ftp_no_timeout (__main__.TimeoutTest) ... ok
test_ftp_timeout (__main__.TimeoutTest) ... ok
test_http_basic (__main__.TimeoutTest) ... ok
test_http_default_timeout (__main__.TimeoutTest) ... ok
test_http_no_timeout (__main__.TimeoutTest) ... ok
test_http_timeout (__main__.TimeoutTest) ... ok
--
Ran 16 tests in 15.259s
OK (skipped=1)
[137983 refs]
--
Added file: http://bugs.python.org/file29511/issue-5550-2.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5550
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options
[issue12455] urllib2 forces title() on header names, breaking some requests
karl added the comment: terry.reedy: You said: and that has_header and get_header *require* the .capitalize() form and reject the .title() form. I made a patch for these two. See http://bugs.python.org/issue5550 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12455 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5550] [urllib.request]: Comparison of HTTP headers should be insensitive to the case
karl added the comment:
The wireshark trace is a different domain than the code example. But let's see.
cocobear added:
headers = [(Content-Type,application/oct-stream),]
with a Content-Type, not the capitalized Type.
BUT in the source code or urllib/request.py there is
http://hg.python.org/cpython/file/3.3/Lib/urllib/request.py#l1184
if not request.has_header('Content-type')
And if you check has_header at
http://hg.python.org/cpython/file/3.3/Lib/urllib/request.py#l367
it compares exactly the string it has received, when HTTP headers are case
insensitive. It reminds me of
http://bugs.python.org/issue12455
http://bugs.python.org/issue17322
with capitalization issues.
I'm changing the title to be more exact.
--
nosy: +karlcow
title: urllib2 use of opener.addheaders - [urllib.request]: Comparison of HTTP
headers should be insensitive to the case
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5550
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error , ability send a detailed response
karl added the comment: Thanks! Yes we should open a separate issue. I will look at the current patch and see what I can propose for the next step. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error , ability send a detailed response
karl added the comment: just a minor issue you made a mistake in the commit message. The issue number is 12921 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17410] Generator-based HTMLParser
karl added the comment: flying sheep: do you plan to make it easier to use the HTML5 algorithm? http://www.w3.org/TR/html5/syntax.html#parsing -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17410] Generator-based HTMLParser
karl added the comment: Ezio: I'm talking about HTML5 Parsing algorithm, not about about parsing html* documents. :) The only python parser I know who is closer of the HTML5 parser algorithm is https://code.google.com/p/html5lib/ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17410 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17340] Handle malformed cookie
karl added the comment: Yes the new RFC has been written by Adam Barth who wanted to describe things matching the reality of HTTP and servers/browsers issues. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17340 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17340] Handle malformed cookie
karl added the comment: The current status of RFC6265 is PROPOSED STANDARD http://www.rfc-editor.org/info/rfc6265 Adam Barth is part of the Google Chrome Team. I do not want to talk for Adam. So better ask him, I don't think he has the energy/will to push further through the IETF process. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17340 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17376] TimedRotatingFileHandler documentation regarding 'Week day' lacking
Changes by karl karl+pythonb...@la-grange.net: -- nosy: +vinay.sajip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17376 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue747320] rfc2822 formatdate functionality duplication
karl added the comment: Ok after comments and review by Eric Araujo on the previous patch. See issue-747320-3.patch I have ran → ./python.exe Lib/test/test_httpservers.py […] -- Ran 39 tests in 3.734s OK [137158 refs] That said there is no specific tests for date_time_string() and log_date_time_string(). That might be something to consider. → grep date_time Lib/test/test_httpservers.py [nil] -- Added file: http://bugs.python.org/file29352/issue-747320-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue747320 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17340] Handle malformed cookie
karl added the comment: Just a quick note that the new specification for HTTP State Mechanism (aka cookies) is http://tools.ietf.org/html/rfc6265 keakon, Do you know why her cookie was ',BRIDGE_R=;' -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17340 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17375] Add docstrings to methods in the threading module
karl added the comment: Here an attempt at fixing it. See issue-17375-1.patch for Python 3.3 Hope it helps. -- keywords: +patch nosy: +karlcow Added file: http://bugs.python.org/file29338/issue-17375-1.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17375 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17375] Add docstrings to methods in the threading module
karl added the comment: Ah bummer! :) it was already done. :) Well it seems already well commented in the review. :) Closing this one as duplicate? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17375 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17376] TimedRotatingFileHandler documentation regarding 'Week day' lacking
karl added the comment: Is it better like this? See the patch. -- keywords: +patch nosy: +karlcow Added file: http://bugs.python.org/file29341/issue-17376-doc-3.3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17376 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17324] SimpleHTTPServer serves files even if the URL has a trailing slash
karl added the comment: orsenthil, would that test work? See issue-17324-test-1.patch Here the result of the test which is FAIL as planned (given the current issue). → ./python.exe Lib/test/test_httpservers.py test_header_buffering_of_send_error (__main__.BaseHTTPRequestHandlerTestCase) ... ok […] test_get (__main__.SimpleHTTPServerTestCase) ... FAIL […] test_start_with_double_slash (__main__.SimpleHTTPRequestHandlerTestCase) ... ok == FAIL: test_get (__main__.SimpleHTTPServerTestCase) -- Traceback (most recent call last): File Lib/test/test_httpservers.py, line 254, in test_get self.check_status_and_reason(response, 404, data=self.data) File Lib/test/test_httpservers.py, line 243, in check_status_and_reason self.assertEqual(response.status, status) AssertionError: 200 != 404 -- Ran 39 tests in 4.382s FAILED (failures=1) Traceback (most recent call last): File Lib/test/test_httpservers.py, line 686, in module test_main() File Lib/test/test_httpservers.py, line 680, in test_main SimpleHTTPRequestHandlerTestCase, File /Users/karl/Documents/2011/cpython/Lib/test/support.py, line 1589, in run_unittest _run_suite(suite) File /Users/karl/Documents/2011/cpython/Lib/test/support.py, line 1564, in _run_suite raise TestFailed(err) test.support.TestFailed: Traceback (most recent call last): File Lib/test/test_httpservers.py, line 254, in test_get self.check_status_and_reason(response, 404, data=self.data) File Lib/test/test_httpservers.py, line 243, in check_status_and_reason self.assertEqual(response.status, status) AssertionError: 200 != 404 [141593 refs] -- keywords: +patch nosy: +karlcow Added file: http://bugs.python.org/file29329/issue-17324-test-1.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17324 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17369] Message.get_filename produces exception if the RFC2231 encoding is ill-formed
karl added the comment: r.david.murray, how did you enter the first without a syntax error? import email.message m = message_from_string(Content-Disposition: attachment; filename*0*=can't decode this filename) File stdin, line 1 m = message_from_string(Content-Disposition: attachment; filename*0*=can't decode this filename) ^ SyntaxError: invalid syntax -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17369 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17138] XPath error in xml.etree.ElementTree
karl added the comment: http://docs.python.org/3/library/xml.etree.elementtree.html#supported-xpath-syntax 20.5.2. XPath support This module provides limited support for XPath expressions for locating elements in a tree. The goal is to support a small subset of the abbreviated syntax; a full XPath engine is outside the scope of the module. What you could do is testing the values with text once you have matched all elements of your choice. import xml.etree.ElementTree as ET xml = ET.fromstring(rooth11/h1ph12/h1/p/root) result = xml.findall(.//h1) for i in result: ... print(i.text) ... 1 2 Could you close the issue? -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17138 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error and trailing newline
karl added the comment: The culprit is here http://hg.python.org/cpython/file/3.3/Lib/http/server.py#l320 That an application or a person decides to send another message is ok. Designer choice. That the library is sending something optional as a test seems more uncomfortable. The list of codes for 4xx is declared at http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-22#section-6.5 In HTTP, only the code is mandatory, the text is optional. But indeed you reveal a loophole, which means that someone might want to send a message for the body. That said, I still do not think it should lend in the header for reasons explained previously (serious breakage). I will come up with something which is fixing the issue without breaking the existent. That said, do I open another bug for the test? The test should be fixed too. It should test 400 only. and not the full status-line. The previous test is also an issue, testing 414. Because the prose in the spec is URI Too Long and not HTTP/1.1 414 Request-URI Too Long http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-22#section-6.5.12 Hmmm… I see all tests are like this. It should be fixed. Opening bug? Thought? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17355] http tests testing more than the error code are fragile
New submission from karl:
Some of the tests of the HTTP Test suite are checking for the full status-line,
instead of just the error code.
Why is it an issue?
1. The only mandatory part in the status-line is the error code. The phrase is
optional. For example the response is made of 3 parts.
HTTP/1.1 400 Bad Request
('HTTP/1.1', '400', 'Bad Request')
Only 400 is the mandatory part of the error code and is testing the error.
2. It creates dependencies on how we conceive, modify the construction of the
status-line and error messages.
3. Some tests are testing messages with different optional texts.
http://hg.python.org/cpython/file/3.3/Lib/test/test_httpservers.py#l640
Test for HTTP/1.1 400 Line Too Long\r\n.
Instead of just testing 400. (A specific message could be put in the body,
but that's informational)
http://hg.python.org/cpython/file/3.3/Lib/test/test_httpservers.py#l633
Test for HTTP/1.1 414 Request-URI Too Long\r\n
instead of just testing 414, btw the spec says for the optional message URI
Too Long
http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-22#section-6.5.12
Testing the message, IMHO, should be only, for testing the construction of the
message. The tests here are made for testing the right error code, but they try
to do more.
Hope it helps :)
Threading: related to http://bugs.python.org/issue12921#msg183520
--
messages: 183521
nosy: karlcow, orsenthil
priority: normal
severity: normal
status: open
title: http tests testing more than the error code are fragile
versions: Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17355
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17355] http tests testing more than the error code are fragile
Changes by karl karl+pythonb...@la-grange.net: -- components: +Library (Lib), Tests ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17355 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error and trailing newline
karl added the comment: Senthil, I created another bug reports for the tests which are fragile. http://bugs.python.org/issue17355 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error and trailing newline
karl added the comment: hehe. No hard feelings. I still do not think it is a good idea to test the error code and its associated message in the same test. :) For example, in RFC2616, 414 is defined as 414 Request-URI Too Long and in the HTTP1.1bis (which will not get a new version number) because the goal of the work was to just clarify and not make incompatible changes, 414 is defined as 414 URI Too Long which is fine because the message is optional. With the current tests, it will make it hard to modify :) http://hg.python.org/cpython/file/3.3/Lib/http/server.py#l627 # More about this specific issue Right now, send_error groks everything, which is not very good in terms of security and side effects. http://hg.python.org/cpython/file/3.3/Lib/http/server.py#l404 def send_error(self, code, message=None): Then later on: try: shortmsg, longmsg = self.responses[code] * shortmsg is supposed to bewhat is written in the spec. * longmsg is specific to the python project. When the message is not defined it takes the shortmsg http://hg.python.org/cpython/file/3.3/Lib/http/server.py#l421 but if defined it sends everything whatever it is http://hg.python.org/cpython/file/3.3/Lib/http/server.py#l428 Checking the status-line http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-22#section-3.1.2 3.1.2. Status Line The first line of a response message is the status-line, consisting of the protocol version, a space (SP), the status code, another space, a possibly-empty textual phrase describing the status code, and ending with CRLF. status-line = HTTP-version SP status-code SP reason-phrase CRLF The status-code element is a 3-digit integer code describing the result of the server's attempt to understand and satisfy the client's corresponding request. The rest of the response message is to be interpreted in light of the semantics defined for that status code. See Section 6 of [Part2] for information about the semantics of status codes, including the classes of status code (indicated by the first digit), the status codes defined by this specification, considerations for the definition of new status codes, and the IANA registry. status-code= 3DIGIT The reason-phrase element exists for the sole purpose of providing a textual description associated with the numeric status code, mostly out of deference to earlier Internet application protocols that were more frequently used with interactive text clients. A client SHOULD ignore the reason-phrase content. reason-phrase = *( HTAB / SP / VCHAR / obs-text ) * So client SHOULD ignore the reason-phrase * The reason-phrase can only contains ** HTAB (horizontal tab) ** SP (space) ** VCHAR (any visible [USASCII] character) ** obs-text *** As a convention, ABNF rule names prefixed with obs- denote obsolete grammar rules that appear for historical reasons. *** obs-text = %x80-FF (range of characters) A hacky way to mitigate the issue would be to 1. extract the first line (stop after the first CRLF) 2. sanitize this line. (allow only *(HTAB / SP / VCHAR / %x80-FF) ) 3. send only this. Thoughts? Honestly, I do not find very satisfying. ;) but at least it should not break everything. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error and trailing newline
karl added the comment: R.david Trying another patch just for understanding if it's what you meant. What it does: 1. adding an 'explain' keyword 2. escaping the explain message for the body 3. checking for injection of newline in the reason phrase. For the part 3, TODO: check if there are other parts in the library if the reason phrase is injected in the message. -- Added file: http://bugs.python.org/file29314/issue-12921-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8732] Should urrllib2.urlopen send an Accept-Encoding header?
karl added the comment: What was the content of http://support.github.com/discussions/site/1510 I can't find it. Is the issue still going on? -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8732 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8450] httplib: false BadStatusLine() raised
karl added the comment: Hmm no code. I wonder if it's about this part. http://hg.python.org/cpython/file/3.3/Lib/http/client.py#l321 -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8450 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13359] urllib2 doesn't escape spaces in http requests
karl added the comment: The issue with the current patch is that it is escaping more than only the spaces, with possibly indirect border effect. Anne van Kesteren is in the process of creating a parsing/writing specification for URL. Not finished but putting it here for future reference. http://url.spec.whatwg.org/ -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13359 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14132] Redirect is not working correctly in urllib2
karl added the comment: → curl -sI http://kniznica.uniza.sk/opac HTTP/1.1 302 Moved Temporarily Date: Wed, 06 Mar 2013 03:23:06 GMT Server: Indy/9.0.50 Content-Type: text/html Location: ?fs=C79F09C9F1304E7AA4FF7C211BEA2B9Bfn=main → python3.3 Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type help, copyright, credits or license for more information. import urllib.parse urllib.parse.urlparse(http://kniznica.uniza.sk/opac;) ParseResult(scheme='http', netloc='kniznica.uniza.sk', path='/opac', params='', query='', fragment='') urllib.parse.urlparse(?fs=C79F09C9F1304E7AA4FF7C211BEA2B9Bfn=main) ParseResult(scheme='', netloc='', path='', params='', query='fs=C79F09C9F1304E7AA4FF7C211BEA2B9Bfn=main', fragment='') Redirection is defined at http://hg.python.org/cpython/file/5e294202f93e/Lib/urllib/request.py#l643 -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14132 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15851] Lib/robotparser.py doesn't accept setting a user agent string, instead it uses the default.
karl added the comment:
Setting a user agent string should be possible.
My guess is that the default library has been used by an abusive client (by
mistake or intent) and wikimedia project has decided to blacklist the client
based on the user-agent string sniffing.
The match is on anything which matches
Python-urllib in UserAgentString
See below:
import urllib.request
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Python-urllib')]
fobj = opener.open('http://en.wikipedia.org/robots.txt')
Traceback (most recent call last):
File stdin, line 1, in module
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 479, in open
response = meth(req, response)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 591, in http_response
'http', request, response, code, msg, hdrs)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 517, in error
return self._call_chain(*args)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 451, in _call_chain
result = func(*args)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 599, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
import urllib.request
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Pythonurllib/3.3')]
fobj = opener.open('http://en.wikipedia.org/robots.txt')
fobj
http.client.HTTPResponse object at 0x101275850
import urllib.request
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Pyt-honurllib/3.3')]
fobj = opener.open('http://en.wikipedia.org/robots.txt')
import urllib.request
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Python-urllib')]
fobj = opener.open('http://en.wikipedia.org/robots.txt')
Traceback (most recent call last):
File stdin, line 1, in module
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 479, in open
response = meth(req, response)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 591, in http_response
'http', request, response, code, msg, hdrs)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 517, in error
return self._call_chain(*args)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 451, in _call_chain
result = func(*args)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py,
line 599, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
import urllib.request
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Python-urlli')]
fobj = opener.open('http://en.wikipedia.org/robots.txt')
Being able to change the header might indeed be a good thing.
--
nosy: +karlcow
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15851
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces (and other weird stuff)
karl added the comment: R. David.: A crazy idea that occurred to me was to create an rfc822-style-header management module, and share it between email, http, and urllib. Yes it is basically what I had in mind when I said: Maybe the way forward in the future is to have a header factory shared by all HTTP libs? I'm not sure if it's easy to share in between emails and HTTP. Emails are now defined by RFC5322: https://tools.ietf.org/html/rfc5322#section-2.2 2.2. Header Fields Header fields are lines beginning with a field name, followed by a colon (:), followed by a field body, and terminated by CRLF. A field name MUST be composed of printable US-ASCII characters (i.e., characters that have values between 33 and 126, inclusive), except colon. A field body may be composed of printable US-ASCII characters as well as the space (SP, ASCII value 32) and horizontal tab (HTAB, ASCII value 9) characters (together known as the white space characters, WSP). A field body MUST NOT include CR and LF except when used in folding and unfolding, as described in section 2.2.3. All field bodies MUST conform to the syntax described in sections 3 and 4 of this specification. Maybe it's doable and worth exploring but I have the feeling we would end up with something along field-name : field-value and all the rules for field-name and field-value being different for HTTP and email, because they really are. Folding, set of characters, etc. :) Another thing which should be also the opportunity for opening another issue. HTTP headers and python dictionaries is not a very good match. :) But this is drifting off-topic. :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17322 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error and trailing newline
karl added the comment: orsenthil, I made a proper patch for it with hg diff. It is very short. See issue-12921-2.patch -- Added file: http://bugs.python.org/file29306/issue-12921-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error and trailing newline
karl added the comment: orsenthil, When you have done a patch for testing I would love to see it. I could not find a proper way of doing it. I'm eager to learn. Thanks. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces (and other weird stuff)
karl added the comment: Hello, So I tested a bit. The production rules defined by the specification are clear. Spaces before and after are forbidden. header-field = field-name : OWS field-value BWS field-name = token field-value= *( field-content / obs-fold ) field-content = *( HTAB / SP / VCHAR / obs-text ) obs-fold = CRLF ( SP / HTAB ) ; obsolete line folding ; see Section 3.2.4 and token = 1*tchar and tchar as tchar = ! / # / $ / % / / ' / * / + / - / . / ^ / _ / ` / | / ~ / DIGIT / ALPHA Here are the production rules for HTTP headers for messages (so both Request and Responses). You can have funky headers, I guess that would be interesting to add to the urllib tests too. Basically to have something in the library, which check if header contains the tchar characters and sends back a warning of exception when not part of it. curl has a bug too, IMHO. Though, one might argue that it is practical for testing bugs. :) On the side of parsing it's clear for the trailing space but unknown for the leading spaces. I sent a long email explaining the issue to the HTTP WG. See http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/1166.html Let's see what will be the answers -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17322 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces (and other weird stuff)
karl added the comment:
OK. I'm inclined to think that we should both remove trailing and leading
spaces/tabs should be removed.
Reasons:
1. Production rules forbid them.
2. Trailing spaces
2.a Conformant servers will ignore with a 400 bad request (opportunity for
another bugs?)
2.b Conformant proxies will rewrite the header by removing spaces.
3. Leading spaces are continuation lines. See below.
I had completely missed that. The syntax for headers is:
header-field = field-name : OWS field-value BWS
field-name = token
field-value= *( field-content / obs-fold )
field-content = *( HTAB / SP / VCHAR / obs-text )
obs-fold = CRLF ( SP / HTAB )
; obsolete line folding
; see Section 3.2.4
obs-fold is about line folding which is basically a header that would look like:
foo: bar\n
blah: something
which could be the equivalent of:
foo: bar blah: something
with foo the header field-name and bar blah: something the header
field-value.
which is clearly not the intent of an author doing:
request.add_header('Accept', 'text/html')
request.add_header(' User-Agent', 'foobar/1.0')
because this would be parsed by a conformant server as
Accept: text/html User-Agent: foobar/1.0
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17322
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces (and other weird stuff)
karl added the comment: R. David Murray, You are right it is not specific to the client library. HTTP headers are part of the message (Request/Response) with both the same constraints. Constraints are put on receivers (receiving a message) and senders (sending a message) of the message (which is not specifically related to client or server). Maybe the way forward in the future is to have a header factory shared by all HTTP libs? I noticed that http.client and http.server had similar issues: in http.server send_header in http.client putheader Which are similar features aka constructing headers for sending with the message. And what would be the elegant way to solve this current bug? Ah… before I forget… The WG is having a meeting in 2 weeks. To make a summary of the HTTPBIS work. See the agenda. http://tools.ietf.org/wg/httpbis/agenda?item=agenda-86-httpbis.html The current documents are in Last Call with no issues unresolved. http://trac.tools.ietf.org/wg/httpbis/trac/report/20 So if R. David is worried that it will change, we can wait a bit more before taking actions, if we are going the way of removing leading/trailing spaces. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17322 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces (and other weird stuff)
karl added the comment: http://hg.python.org/cpython/file/3.3/Lib/urllib/request.py#l359 def add_header(self, key, val): # useful for something like authentication self.headers[key.capitalize()] = val and http://hg.python.org/cpython/file/3.3/Lib/urllib/request.py#l271 in __init__ of class Request: for key, value in headers.items(): self.add_header(key, value) Tests seem to be there, but there are none. Is there a reason why there are no tests? http://hg.python.org/cpython/file/3.3/Lib/test/test_urllib2.py#l98 Or should I add a test in http://hg.python.org/cpython/file/3.3/Lib/test/test_urllib.py I'm confused :) I need guidance. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17322 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces (and other weird stuff)
karl added the comment: I created 4 tests for testing trailing and leading spaces on * add_unredirected_header() * add_header() and modified the functions. Tests passed. → ./python.exe Lib/test/test_urllib2net.py […] test_headers_with_spaces (__main__.OtherNetworkTests) ... ok […] -- Ran 16 tests in 17.148s OK (skipped=1) [137988 refs] See the patch issue-17322-1.patch -- keywords: +patch Added file: http://bugs.python.org/file29292/issue-17322-1.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17322 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11448] docs for HTTPConnection.set_tunnel are ambiguous
karl added the comment: Ah thanks Eric, I will fix that. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11448 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11448] docs for HTTPConnection.set_tunnel are ambiguous
karl added the comment: ok made a proper patch on the rst file with hg diff. See issue-11448-1.patch -- Added file: http://bugs.python.org/file29293/issue-11448-1.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11448 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12455] urllib2 forces title() on header names, breaking some requests
karl added the comment: Are there issues related to removing the capitalize() and title() appears? # title() * http://hg.python.org/cpython/file/886df716cd09/Lib/urllib/request.py#l1239 # capitalize() * http://hg.python.org/cpython/file/886df716cd09/Lib/urllib/request.py#l359 * http://hg.python.org/cpython/file/886df716cd09/Lib/urllib/request.py#l363 * http://hg.python.org/cpython/file/886df716cd09/Lib/urllib/request.py#l1206 Because the behavior is inconsistent, I would live to propose a patch removing them and be sure to be completely neutral with regards to them. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12455 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12455] urllib2 forces title() on header names, breaking some requests
karl added the comment: tests in http://hg.python.org/cpython/file/886df716cd09/Lib/test/test_wsgiref.py#l370 also checking that everything is case insensitive. And the method to get the headers in wsgiref, make sure they are lower-case http://hg.python.org/cpython/file/886df716cd09/Lib/wsgiref/headers.py#l82 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12455 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17307] HTTP PUT request Example
karl added the comment: Sentil: About the PUT/POST, I would say: A POST and PUT methods differs only by the intent of the enclosed representation. In the case of a POST, the target resource (URI) on the server decides what is the meaning of the enclosed representation in the POST request. In a PUT request, the enclosed representation is meant to replace the state of the target resource (URI) on the server. It is why PUT is idempotent. About the code: * http.client I would remove the following comment, because the term file is confusing in HTTP terms. # This will create a resource file with contents of BODY or I would say more exactly # This creates an HTTP message # with the content of BODY as the enclosed representation # for the resource http://localhost:8080/foobar import http.client BODY = Some data conn = http.client.HTTPConnection(localhost, 8080) conn.request(PUT, /foobar, BODY) # The actual PUT Request response = conn.getresponse() print(resp.status, response.reason) Maybe it would be good to display the message which is sent so people can understand what goes on the wire. * urllib the client code for urllib doesn't create challenge, I would had a content-type but no hard feeling about it. On the other hand the server code makes me a bit uncomfortable. It sets people into believing that this is the way you should reply to a PUT which is not really the case. 1. If the resource was not existing and has been successfully created, the server MUST answer 204. 2. if the resource already exists and has been successfully replaced/modified, then the server SHOULD answer either 200 or 204 (depending on the design choice) There are plenty of other cases depending on the constraints. See for the details http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-22#section-4.3.4 If we keep the server code, I would be willing to have a note saying that it is not usable as-is in production code. Does that make sense? :) -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12455] urllib2 forces title() on header names, breaking some requests
karl added the comment: Note that HTTP header fields are case-insensitive. See http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging#section-3.2 Each HTTP header field consists of a case-insensitive field name followed by a colon (:), optional whitespace, and the field value. Basically the author of a request can set them to whatever he/she wants. But we should, IMHO, respect the author intent. It might happen that someone will choose a specific combination of casing to deal with broken servers and/or proxies. So a cycle of set/get/send should not modify at all the headers. -- nosy: +karlcow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12455 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces
New submission from karl:
For HTTP header field names parsing, see
http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-22#section-3.2.4
No whitespace is allowed between the header field-name and colon. In
the past, differences in the handling of such whitespace have led to
security vulnerabilities in request routing and response handling. A
server MUST reject any received request message that contains
whitespace between a header field-name and colon with a response code
of 400 (Bad Request). A proxy MUST remove any such whitespace from a
response message before forwarding the message downstream.
In python3.3 currently
import urllib.request
req = urllib.request.Request('http://www.example.com/')
req.add_header('FoO ', 'Yeah')
req.header_items()
[('Foo ', 'Yeah'), ('User-agent', 'Python-urllib/3.3'), ('Host',
'www.example.com')]
The space has not been removed. So we should fix that at least. This is a bug.
I'm not familiar with the specific security issues mentioned in the spec.
Note that many things can be done too: :/
req.add_header('FoO \n blah', 'Yeah')
req.add_header('Foo:Bar\nFoo2', 'Yeah')
req.header_items()
[('Foo:bar\nfoo2', 'Yeah'), ('Foo \n blah', 'Yeah'), ('Foo ', 'Yeah'),
('User-agent', 'Python-urllib/3.3'), ('Host', 'www.example.com')]
I will check for making a patch tomorrow.
--
components: Library (Lib)
messages: 183234
nosy: karlcow, orsenthil
priority: normal
severity: normal
status: open
title: urllib.request add_header() currently allows trailing spaces
versions: Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17322
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces (and other weird stuff)
Changes by karl karl+pythonb...@la-grange.net: -- title: urllib.request add_header() currently allows trailing spaces - urllib.request add_header() currently allows trailing spaces (and other weird stuff) ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17322 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17322] urllib.request add_header() currently allows trailing spaces (and other weird stuff)
karl added the comment:
Yet another one leading spaces :(
req = urllib.request.Request('http://www.example.com/')
req.header_items()
[]
req.add_header(' Foo3', 'Ooops')
req.header_items()
[(' foo3', 'Ooops')]
req.headers
{' foo3': 'Ooops'}
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17322
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12455] urllib2 forces title() on header names, breaking some requests
karl added the comment: So looking at the casing of headers, I discovered other issues. I opened another bug. http://bugs.python.org/issue17322 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12455 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue747320] rfc2822 formatdate functionality duplication
karl added the comment: R. David Murray: Sure. Is it better? issue-747320-1.patch It seems there are already tests for gmt date format in Lib/test/test_email/test_utils.py -- Added file: http://bugs.python.org/file29279/issue-747320-1.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue747320 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17302] HTTP/2.0 - Implementations/Testing efforts
karl added the comment: Read the thread. Thanks Antoine. Better understanding. I'm still discovering how the community is really working. Trying to fix a few things in the mean time http://bugs.python.org/issue12921 http://bugs.python.org/issue747320 http://bugs.python.org/issue11448 http://bugs.python.org/issue7370 (maybe this one is a duplicate of 747320) This one http://bugs.python.org/issue15799 which is still open, make me thinks, that it might in the new class to have things for strict production rules, and some parsing rules with a warning mode, if the user cares. Thanks again for the context Antoine. Maybe we should close this bug as postponed? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17302 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17319] http.server.BaseHTTPRequestHandler send_response_only doesn't check the type and value of the code.
New submission from karl:
def send_response_only(self, code, message=None):
http://hg.python.org/cpython/file/3.3/Lib/http/server.py#l448
There is no type checking on code or if the code is appropriate. Let's take
==
#!/usr/bin/env python3.3
import http.server
class HTTPHandler(http.server.BaseHTTPRequestHandler):
A very simple server
def do_GET(self):
self.send_response(200)
self.send_header('Content-type', 'text/plain')
self.end_headers()
self.wfile.write(bytes('Response body\n\n', 'latin1'))
if __name__ == '__main__':
addr = ('', 9000)
http.server.HTTPServer(addr, HTTPHandler).serve_forever()
=
A request is working well.
=
→ http GET localhost:9000
HTTP/1.0 200 OK
Server: BaseHTTP/0.6 Python/3.3.0
Date: Thu, 28 Feb 2013 04:00:44 GMT
Content-type: text/plain
Response body
=
And the server log is
127.0.0.1 - - [27/Feb/2013 23:00:44] GET / HTTP/1.1 200 -
Then let's try
=
#!/usr/bin/env python3.3
import http.server
class HTTPHandler(http.server.BaseHTTPRequestHandler):
A very simple server
def do_GET(self):
self.send_response(999)
self.send_header('Content-type', 'text/plain')
self.end_headers()
self.wfile.write(bytes('Response body\n\n', 'latin1'))
if __name__ == '__main__':
addr = ('', 9000)
http.server.HTTPServer(addr, HTTPHandler).serve_forever()
=
The response is
=
→ http GET localhost:9000
HTTP/1.0 999
Server: BaseHTTP/0.6 Python/3.3.0
Date: Thu, 28 Feb 2013 03:55:54 GMT
Content-type: text/plain
Response body
=
and the log server is
127.0.0.1 - - [27/Feb/2013 22:55:12] GET / HTTP/1.1 999 -
And finally
=
#!/usr/bin/env python3.3
import http.server
class HTTPHandler(http.server.BaseHTTPRequestHandler):
A very simple server
def do_GET(self):
self.send_response('foobar')
self.send_header('Content-type', 'text/plain')
self.end_headers()
self.wfile.write(bytes('Response body\n\n', 'latin1'))
if __name__ == '__main__':
addr = ('', 9000)
http.server.HTTPServer(addr, HTTPHandler).serve_forever()
=
The response is then
=
→ http GET localhost:9000
HTTPConnectionPool(host='localhost', port=9000): Max retries exceeded with url:
/
=
and the server log is
127.0.0.1 - - [27/Feb/2013 22:56:39] GET / HTTP/1.1 foobar -
Exception happened during processing of request from ('127.0.0.1', 53917)
Traceback (most recent call last):
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/socketserver.py,
line 306, in _handle_request_noblock
self.process_request(request, client_address)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/socketserver.py,
line 332, in process_request
self.finish_request(request, client_address)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/socketserver.py,
line 345, in finish_request
self.RequestHandlerClass(request, client_address, self)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/socketserver.py,
line 666, in __init__
self.handle()
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/http/server.py,
line 400, in handle
self.handle_one_request()
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/http/server.py,
line 388, in handle_one_request
method()
File ../25/server.py, line 8, in do_GET
self.send_response('foobar')
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/http/server.py,
line 444, in send_response
self.send_response_only(code, message)
File
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/http/server.py,
line 459, in send_response_only
(self.protocol_version, code, message)).encode(
TypeError: %d format: a number is required, not str
Both error messages and server logs are not very helpful.
Shall we fix it?
What others think?
I guess there should be test cases too?
I'm happy to make unit test cases for it though I might need a bit of guidance
as I'm not comfortable with unit test cases mocking connections.
--
components: Library (Lib)
messages: 183201
nosy: karlcow, orsenthil
priority: normal
severity: normal
status: open
title: http.server.BaseHTTPRequestHandler send_response_only doesn't check the
type and value of the code.
type: enhancement
versions: Python 3.3
[issue17302] HTTP/2.0 - Implementations/Testing efforts
New submission from karl: Are there plans to develop an HTTP/2.0 library in parallel of the specification development? It will not be ready before years, but it would be good to have an evolving implementation. Or should it be done outside of python.org? Reference: https://github.com/http2 -- components: Library (Lib) messages: 183086 nosy: karlcow priority: normal severity: normal status: open title: HTTP/2.0 - Implementations/Testing efforts versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17302 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17302] HTTP/2.0 - Implementations/Testing efforts
karl added the comment: agreed on HTTP/1.1, is there a plan to fix it too ;) because the current http.server seems to be untouchable without breaking stuff all around :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17302 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error and trailing newline
karl added the comment:
Testing your code in Listing 1
→ curl -sI http://localhost:9000/
HTTP/1.0 501 Unsupported method ('HEAD')
Server: BaseHTTP/0.6 Python/3.3.0
Date: Tue, 26 Feb 2013 23:38:32 GMT
Content-Type: text/html;charset=utf-8
Connection: close
So this is normal,
http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-22#section-6.6.2
except that it would be better to use 501 Not Implemented through the prose
is optional. The content-type is also kind of useless. That would deserve to
open another bug.
And
→ curl http://localhost:9000/
Server: BaseHTTP/0.6 Python/3.3.0
Date: Tue, 26 Feb 2013 23:39:46 GMT
Content-Type: text/html;charset=utf-8
Connection: close
!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd;
html
head
meta http-equiv=Content-Type content=text/html;charset=utf-8
titleError response/title
/head
body
h1Error response/h1
pError code: 500/p
pMessage: Traceback (most recent call last):
File server.py, line 9, in do_GET
assert(False)
AssertionError
./p
pError code explanation: 500 - Server got itself in trouble./p
/body
/html
OK. The server is answering with HTTP/1.0 and then a Traceback… which has
nothing to do here.
We can see that in more details with a telnet
→ telnet localhost 9000
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1
Host: localhost:9000
HTTP/1.0 500 Traceback (most recent call last):
File server.py, line 9, in do_GET
assert(False)
AssertionError
Server: BaseHTTP/0.6 Python/3.3.0
Date: Tue, 26 Feb 2013 23:49:04 GMT
Content-Type: text/html;charset=utf-8
Connection: close
!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd;
html
head
meta http-equiv=Content-Type content=text/html;charset=utf-8
titleError response/title
/head
body
h1Error response/h1
pError code: 500/p
pMessage: Traceback (most recent call last):
File server.py, line 9, in do_GET
assert(False)
AssertionError
./p
pError code explanation: 500 - Server got itself in trouble./p
/body
/html
Note that when not sending the traceback with the following code
#!/usr/bin/env python3.3
import http.server
import traceback
class httphandler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
try:
assert(False)
except:
self.send_error(500)
if __name__=='__main__':
addr=('',9000)
http.server.HTTPServer(addr,httphandler).serve_forever()
Everything is working well.
→ telnet localhost 9000
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1
Host: localhost:9000
HTTP/1.0 500 Internal Server Error
Server: BaseHTTP/0.6 Python/3.3.0
Date: Tue, 26 Feb 2013 23:51:46 GMT
Content-Type: text/html;charset=utf-8
Connection: close
!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd;
html
head
meta http-equiv=Content-Type content=text/html;charset=utf-8
titleError response/title
/head
body
h1Error response/h1
pError code: 500/p
pMessage: Internal Server Error./p
pError code explanation: 500 - Server got itself in trouble./p
/body
/html
Connection closed by foreign host.
I'm looking at http://hg.python.org/cpython/file/3.3/Lib/http/server.py#l404
For the second part of your message. I don't think the two issues should be
mixed. Maybe open another bug report.
--
nosy: +karlcow
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12921
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error and trailing newline
karl added the comment: OK I had understand a bit better. self.send_error(code, msg) is used for * The body * The HTTP header * and the log That's bad, very bad. I do not think it should be used for the HTTP Header at all. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12921] http.server.BaseHTTPRequestHandler.send_error and trailing newline
karl added the comment: ok I modify the code of server.py so that the server doesn't send the private message but the one which is already assigned by the library as it should. If there is a need for customization, there should be two separate variables, but which could lead to the same issues. After modifications this is what I get. → telnet localhost 9000 Trying ::1... telnet: connect to address ::1: Connection refused Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GET / HTTP/1.1 Host: localhost:9000 HTTP/1.0 500 Internal Server Error Server: BaseHTTP/0.6 Python/3.3.0 Date: Wed, 27 Feb 2013 00:21:21 GMT Content-Type: text/html;charset=utf-8 Connection: close !DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01//EN http://www.w3.org/TR/html4/strict.dtd; html head meta http-equiv=Content-Type content=text/html;charset=utf-8 titleError response/title /head body h1Error response/h1 pError code: 500/p pMessage: Traceback (most recent call last): File server.py, line 11, in do_GET assert(False) AssertionError ./p pError code explanation: 500 - Server got itself in trouble./p /body /html Connection closed by foreign host. I joined the patch: server.issue12921.patch -- keywords: +patch Added file: http://bugs.python.org/file29255/server.issue12921.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12921 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
