mspagon commented on issue #1487: URL: https://github.com/apache/libcloud/issues/1487#issuecomment-689757521
I traced the bug down to a single-line of code. A GET request is sent with an empty body `''` incorrectly as a **chunked** request and the `0` appears because it is supposed to signal the final chunk. ## The Line of Code The code is in `libcloud/http.py` and relates to how `requests` is being used to prepare requests and potentially affects every HTTP request libcloud makes. A `PreparedRequest` is being constructed and then its body is being overwritten. The assignment seems redundant to me, as `Requests` does the job of translating it. [Line 244 of libcloud/http.py](https://github.com/apache/libcloud/blob/72de440a508fdb4ea546d4e586a452472f1cbd5d/libcloud/http.py#L244) **(LibcloudConnection class)** ``` def prepared_request(self, method, url, body=None, headers=None, raw=False, stream=False): headers = self._normalize_headers(headers=headers) req = requests.Request(method, ''.join([self.host, url]), data=body, headers=headers) prepped = self.session.prepare_request(req) prepped.body = body # <<<<<< This line. self.response = self.session.send( prepped, stream=stream, verify=self.ca_cert if self.ca_cert is not None else self.verify) ``` ## Why it's the Problem When a request is prepared, if its body is an empty string `''`, the preparation step ignores the body and sets the resulting `PreparedRequest` body to `None`. Subsequently setting the body back to an empty string `''` breaks requests logic when sending the request. `Requests` now thinks the request should be chunked because `body=''` breaks this logic: [Line 420 of requests/adapters.py](https://github.com/psf/requests/blob/master/requests/adapters.py#L420) ``` chunked = not (request.body is None or 'Content-Length' in request.headers) ``` A raw HTTP chunked request is sent with no body. As a result only a `0` is transmitted to signify the last chunk. This coincides with when our server receives the HTTP 400 error. [Line 469 of requests/adapters.py](https://github.com/psf/requests/blob/master/requests/adapters.py#L469) ``` for i in request.body: low_conn.send(hex(len(i))[2:].encode('utf-8')) low_conn.send(b'\r\n') low_conn.send(i) low_conn.send(b'\r\n') low_conn.send(b'0\r\n\r\n') ``` ## Issue Appears when Default Value is `''` This problem originates from the default value for the `data` parameter within a provider's connection class `request` method. Some connection classes set the default value for data as `None` instead of `''`, and `None` does not trigger this issue. ``` #libcloud/common/openstack.py OpenStackBaseConnection def request(self, action, params=None, data='', headers=None, ...) #libcloud/common/base.py Connection def request(self, action, params=None, data=None, headers=None, ...) ``` ## Replicating the Behavior You can use this snippet to reproduce the same behavior outside the context of Libcloud. ``` import requests session = requests.Session() req = requests.Request('GET', 'http://www.google.com', data='') # prepare_request() will ignore the empty string and set prepped.body to None. prepped = session.prepare_request(req) # Overwriting None with '' will break requests logic for sending requests. prepped.body = '' # Chunked request is sent with a zero length body. Only terminating 0 is sent. response = session.send(prepped, stream=False, verify=False) ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
