[issue39875] urllib.request.urlopen sends POST data as query string

henrik242 added the comment: Solved! The problem was Solr which it has special handling of POSTed data with the User-Agent starts with 'curl/':

[issue39875] urllib.request.urlopen sends POST data as query string

henrik242 added the comment: Root cause for this seems to be https://bugs.python.org/issue12849 -- ___ Python tracker ___ ___

[issue39875] urllib.request.urlopen sends POST data as query string

henrik242 added the comment: Here's the wireshark output. It seems that urllib adds a "Connection: close" which curl doesn't. Solr doesn't seem to like that. Curl message: POST /solr/my_coll/update?commit=true HTTP/1.1 Host: solr.example.no:12699 User-Agent: curl/7.64.1 Accept: */*

[issue39875] urllib.request.urlopen sends POST data as query string

henrik242 added the comment: The following gives the same failing result too :( >>> import urllib.request >>> data = 'KEY__9927.1>> name="value">{"result":0,"jobId":"9459695","jobNumber":"9927.1"}' >>> url = 'http://solr.example.no:12699/solr/my_coll/update?commit=true' >>> res =

[issue39875] urllib.request.urlopen sends POST data as query string

henrik242 added the comment: Also, it seems that urllib.urlopen just creates a similar Request object when given a data paramenter: def open(self, fullurl, data=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT): # accept a URL or a Request object if isinstance(fullurl, str):

[issue39875] urllib.request.urlopen sends POST data as query string

henrik242 added the comment: Further: method should be a string that indicates the HTTP request method that will be used (e.g. 'HEAD'). If provided, its value is stored in the method attribute and is used by get_method(). The default is 'GET' if data is None or 'POST' otherwise.

[issue39875] urllib.request.urlopen sends POST data as query string

henrik242 added the comment: But why can't the payload be in the Request object? >From the api docs: class urllib.request.Request(url, data=None, headers={}, origin_req_host=None, unverifiable=False, method=None) data must be an object specifying additional data to send to the

[issue39875] urllib.request.urlopen sends POST data as query string

Adrian Petrescu added the comment: (Oops, that was a bad paste! I meant this link: https://docs.python.org/2/library/urllib.html#urllib.urlopen) -- ___ Python tracker ___

[issue39875] urllib.request.urlopen sends POST data as query string

Adrian Petrescu added the comment: This is not a bug, you've just misunderstood the urllib API. If you want to pass POST data as a payload, it's the second `data` parameter to `urlopen`: https://bugs.python.org/?@action=confrego=KX9AqsI0JnOLkplIY1AGKXAmDKa38COy -- nosy: +apetresc

[issue39875] urllib.request.urlopen sends POST data as query string

New submission from henrik242 : curl correctly posts data to Solr: $ curl -v 'http://solr.example.no:12699/solr/my_coll/update?commit=true' \ --data 'KEY__9927.1\ {"result":0,"jobId":"9459695","jobNumber":"9927.1"}' The solr query log says: [20200306T111354,131] [my_coll_shard1_replica_n85]