[issue12849] Cannot override 'connection: close' in urllib2 headers

2020-03-09 Thread henrik242
henrik242 added the comment: Correction: My problem in Issue 39875 was not related to Connection: Close, but with weird POST handling in Solr. -- ___ Python tracker <https://bugs.python.org/issue12

[issue39875] urllib.request.urlopen sends POST data as query string

2020-03-09 Thread henrik242
henrik242 added the comment: Solved! The problem was Solr which it has special handling of POSTed data with the User-Agent starts with 'curl/': https://github.com/apache/lucene-solr/blob/40661489cd590947f513e553a20707d0c82b82e5/solr/core/src/java/org/apache/solr/servlet

[issue39875] urllib.request.urlopen sends POST data as query string

2020-03-06 Thread henrik242
henrik242 added the comment: Root cause for this seems to be https://bugs.python.org/issue12849 -- ___ Python tracker <https://bugs.python.org/issue39

[issue12849] Cannot override 'connection: close' in urllib2 headers

2020-03-06 Thread henrik242
henrik242 added the comment: That mandatory "Connection: close" makes it impossible to POST a data request to Solr, as described in https://bugs.python.org/issue39875 It would be very helpful if it could be made optional. -- nosy: +henrik242 versions: +

[issue39875] urllib.request.urlopen sends POST data as query string

2020-03-06 Thread henrik242
henrik242 added the comment: Here's the wireshark output. It seems that urllib adds a "Connection: close" which curl doesn't. Solr doesn't seem to like that. Curl message: POST /solr/my_coll/update?commit=true HTTP/1.1 Host: solr.example.no:12699 User-Agent: curl/7.6

[issue39875] urllib.request.urlopen sends POST data as query string

2020-03-06 Thread henrik242
henrik242 added the comment: The following gives the same failing result too :( >>> import urllib.request >>> data = 'KEY__9927.1>> name="value">{"result":0,"jobId":"9459695","jobNumber":"9927.1&q

[issue39875] urllib.request.urlopen sends POST data as query string

2020-03-06 Thread henrik242
henrik242 added the comment: Also, it seems that urllib.urlopen just creates a similar Request object when given a data paramenter: def open(self, fullurl, data=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT): # accept a URL or a Request object if isinstance(fullurl, str

[issue39875] urllib.request.urlopen sends POST data as query string

2020-03-06 Thread henrik242
henrik242 added the comment: Further: method should be a string that indicates the HTTP request method that will be used (e.g. 'HEAD'). If provided, its value is stored in the method attribute and is used by get_method(). The default is 'GET' if data is None or 'POST' otherwise

[issue39875] urllib.request.urlopen sends POST data as query string

2020-03-06 Thread henrik242
henrik242 added the comment: But why can't the payload be in the Request object? >From the api docs: class urllib.request.Request(url, data=None, headers={}, origin_req_host=None, unverifiable=False, method=None) data must be an object specifying additional data to s

[issue39875] urllib.request.urlopen sends POST data as query string

2020-03-06 Thread henrik242
New submission from henrik242 : curl correctly posts data to Solr: $ curl -v 'http://solr.example.no:12699/solr/my_coll/update?commit=true' \ --data 'KEY__9927.1\ {"result":0,"jobId":"9459695","jobNumber":"9927.1"}' The solr query log says: