Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package python-h11 for openSUSE:Factory checked in at 2021-02-07 15:21:40 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-h11 (Old) and /work/SRC/openSUSE:Factory/.python-h11.new.28504 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-h11" Sun Feb 7 15:21:40 2021 rev:8 rq:869819 version:0.12.0 Changes: -------- --- /work/SRC/openSUSE:Factory/python-h11/python-h11.changes 2020-11-10 13:53:31.658841567 +0100 +++ /work/SRC/openSUSE:Factory/.python-h11.new.28504/python-h11.changes 2021-02-07 15:24:16.218216392 +0100 @@ -1,0 +2,17 @@ +Fri Feb 5 17:05:15 UTC 2021 - Luigi Baldoni <[email protected]> + +- Update to version to 0.12.0 + * Add early detection of invalid http data when request line + starts with binary + * Drop support for Python 2 + * Fix ReST formatting + * Tuned maybe_extract_next_line to search only \r\n + * Changed the ReceiveBuffer + * Speed up maybe_extract_lines and removed unused variables + * Changed the maybe_extract_lines logic according PR review + * Small rfg (renamed body_and_headers_delimiter_regex -> + blank_line_delimiiter_regex) and slightly updated docs + * Fixed some performance issues + * Added ability to use LF, not only CRLF delimiter + +------------------------------------------------------------------- Old: ---- h11-0.11.0.tar.gz New: ---- h11-0.12.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-h11.spec ++++++ --- /var/tmp/diff_new_pack.Bd3kA1/_old 2021-02-07 15:24:16.738216986 +0100 +++ /var/tmp/diff_new_pack.Bd3kA1/_new 2021-02-07 15:24:16.742216991 +0100 @@ -1,7 +1,7 @@ # # spec file for package python-h11 # -# Copyright (c) 2020 SUSE LLC +# Copyright (c) 2021 SUSE LLC # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -17,8 +17,9 @@ %{?!python_module:%define python_module() python-%{**} python3-%{**}} +%define skip_python2 1 Name: python-h11 -Version: 0.11.0 +Version: 0.12.0 Release: 0 Summary: A pure-Python, bring-your-own-I/O implementation of HTTP/11 License: MIT ++++++ h11-0.11.0.tar.gz -> h11-0.12.0.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/PKG-INFO new/h11-0.12.0/PKG-INFO --- old/h11-0.11.0/PKG-INFO 2020-10-05 20:23:10.682039700 +0200 +++ new/h11-0.12.0/PKG-INFO 2021-01-01 12:34:42.171725700 +0100 @@ -1,6 +1,6 @@ -Metadata-Version: 1.1 +Metadata-Version: 1.2 Name: h11 -Version: 0.11.0 +Version: 0.12.0 Summary: A pure-Python, bring-your-own-I/O implementation of HTTP/1.1 Home-page: https://github.com/python-hyper/h11 Author: Nathaniel J. Smith @@ -120,7 +120,8 @@ It has a test suite with 100.0% coverage for both statements and branches. - Currently it supports Python 3 (testing on 3.5-3.8), Python 2.7, and PyPy. + Currently it supports Python 3 (testing on 3.6-3.9) and PyPy 3. + The last Python 2-compatible version was h11 0.11.x. (Originally it had a Cython wrapper for `http-parser <https://github.com/nodejs/http-parser>`_ and a beautiful nested state machine implemented with ``yield from`` to postprocess the output. But @@ -180,12 +181,12 @@ Classifier: License :: OSI Approved :: MIT License Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy -Classifier: Programming Language :: Python :: 2 -Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 -Classifier: Programming Language :: Python :: 3.5 +Classifier: Programming Language :: Python :: 3 :: Only Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: 3.8 +Classifier: Programming Language :: Python :: 3.9 Classifier: Topic :: Internet :: WWW/HTTP Classifier: Topic :: System :: Networking +Requires-Python: >=3.6 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/README.rst new/h11-0.12.0/README.rst --- old/h11-0.11.0/README.rst 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/README.rst 2021-01-01 12:34:33.000000000 +0100 @@ -112,7 +112,8 @@ It has a test suite with 100.0% coverage for both statements and branches. -Currently it supports Python 3 (testing on 3.5-3.8), Python 2.7, and PyPy. +Currently it supports Python 3 (testing on 3.6-3.9) and PyPy 3. +The last Python 2-compatible version was h11 0.11.x. (Originally it had a Cython wrapper for `http-parser <https://github.com/nodejs/http-parser>`_ and a beautiful nested state machine implemented with ``yield from`` to postprocess the output. But diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/docs/source/changes.rst new/h11-0.12.0/docs/source/changes.rst --- old/h11-0.11.0/docs/source/changes.rst 2020-10-05 20:22:54.000000000 +0200 +++ new/h11-0.12.0/docs/source/changes.rst 2021-01-01 12:34:33.000000000 +0100 @@ -5,6 +5,30 @@ .. towncrier release notes start +v0.12.0 (2021-01-01) +-------------------- + +Features +~~~~~~~~ + +- Added support for servers with broken line endings. + + After this change h11 accepts both ``\r\n`` and ``\n`` as a headers + delimiter. (`#7 <https://github.com/python-hyper/h11/issues/7>`__) +- Add early detection of invalid http data when request line starts + with binary (`#122 + <https://github.com/python-hyper/h11/issues/122>`__) + + +Deprecations and Removals +~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Python 2.7 and PyPy 2 support is removed. h11 now requires + Python>=3.6 including PyPy 3. Users running `pip install h11` on + Python 2 will automatically get the last Python 2-compatible + version. (`#114 <https://github.com/python-hyper/h11/issues/114>`__) + + v0.11.0 (2020-10-05) -------------------- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/docs/source/index.rst new/h11-0.12.0/docs/source/index.rst --- old/h11-0.11.0/docs/source/index.rst 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/docs/source/index.rst 2021-01-01 12:34:33.000000000 +0100 @@ -44,7 +44,9 @@ Vital statistics ---------------- -* Requirements: Python 2.7 or Python 3.5+ (PyPy works great) +* Requirements: Python 3.6+ (PyPy works great) + + The last Python 2-compatible version was h11 0.11.x. * Install: ``pip install h11`` diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/fuzz/afl-server.py new/h11-0.12.0/fuzz/afl-server.py --- old/h11-0.11.0/fuzz/afl-server.py 2020-10-05 20:22:54.000000000 +0200 +++ new/h11-0.12.0/fuzz/afl-server.py 2021-01-01 12:34:33.000000000 +0100 @@ -9,11 +9,6 @@ import h11 -if sys.version_info[0] >= 3: - in_file = sys.stdin.detach() -else: - in_file = sys.stdin - def process_all(c): while True: @@ -26,7 +21,7 @@ afl.init() -data = in_file.read() +data = sys.stdin.detach().read() # one big chunk server1 = h11.Connection(h11.SERVER) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/_connection.py new/h11-0.12.0/h11/_connection.py --- old/h11-0.11.0/h11/_connection.py 2020-10-05 20:22:54.000000000 +0200 +++ new/h11-0.12.0/h11/_connection.py 2021-01-01 12:34:33.000000000 +0100 @@ -109,7 +109,7 @@ ################################################################ -class Connection(object): +class Connection: """An object encapsulating the state of an HTTP connection. Args: @@ -425,7 +425,6 @@ event = self._extract_next_receive_event() if event not in [NEED_DATA, PAUSED]: self._process_event(self.their_role, event) - self._receive_buffer.compress() if event is NEED_DATA: if len(self._receive_buffer) > self._max_incomplete_event_size: # 431 is "Request header fields too large" which is pretty diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/_events.py new/h11-0.12.0/h11/_events.py --- old/h11-0.11.0/h11/_events.py 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/h11/_events.py 2021-01-01 12:34:33.000000000 +0100 @@ -24,7 +24,7 @@ request_target_re = re.compile(request_target.encode("ascii")) -class _EventBundle(object): +class _EventBundle: _fields = [] _defaults = {} @@ -85,9 +85,6 @@ def __eq__(self, other): return self.__class__ == other.__class__ and self.__dict__ == other.__dict__ - def __ne__(self, other): - return not self.__eq__(other) - # This is an unhashable type. __hash__ = None diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/_headers.py new/h11-0.12.0/h11/_headers.py --- old/h11-0.11.0/h11/_headers.py 2020-10-05 20:22:54.000000000 +0200 +++ new/h11-0.12.0/h11/_headers.py 2021-01-01 12:34:33.000000000 +0100 @@ -132,7 +132,7 @@ raw_name = name name = name.lower() if name == b"content-length": - lengths = set(length.strip() for length in value.split(b",")) + lengths = {length.strip() for length in value.split(b",")} if len(lengths) != 1: raise LocalProtocolError("conflicting Content-Length headers") value = lengths.pop() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/_readers.py new/h11-0.12.0/h11/_readers.py --- old/h11-0.11.0/h11/_readers.py 2020-10-05 20:22:54.000000000 +0200 +++ new/h11-0.12.0/h11/_readers.py 2021-01-01 12:34:33.000000000 +0100 @@ -54,13 +54,7 @@ def _decode_header_lines(lines): for line in _obsolete_line_fold(lines): - # _obsolete_line_fold yields either bytearray or bytes objects. On - # Python 3, validate() takes either and returns matches as bytes. But - # on Python 2, validate can return matches as bytearrays, so we have - # to explicitly cast back. - matches = validate( - header_field_re, bytes(line), "illegal header line: {!r}", bytes(line) - ) + matches = validate(header_field_re, line, "illegal header line: {!r}", line) yield (matches["field_name"], matches["field_value"]) @@ -70,6 +64,8 @@ def maybe_read_from_IDLE_client(buf): lines = buf.maybe_extract_lines() if lines is None: + if buf.is_next_line_obviously_invalid_request_line(): + raise LocalProtocolError("illegal request line") return None if not lines: raise LocalProtocolError("no request line received") @@ -87,6 +83,8 @@ def maybe_read_from_SEND_RESPONSE_server(buf): lines = buf.maybe_extract_lines() if lines is None: + if buf.is_next_line_obviously_invalid_request_line(): + raise LocalProtocolError("illegal request line") return None if not lines: raise LocalProtocolError("no response line received") @@ -127,7 +125,7 @@ chunk_header_re = re.compile(chunk_header.encode("ascii")) -class ChunkedReader(object): +class ChunkedReader: def __init__(self): self._bytes_in_chunk = 0 # After reading a chunk, we have to throw away the trailing \r\n; if @@ -153,7 +151,7 @@ assert self._bytes_to_discard == 0 if self._bytes_in_chunk == 0: # We need to refill our chunk count - chunk_header = buf.maybe_extract_until_next(b"\r\n") + chunk_header = buf.maybe_extract_next_line() if chunk_header is None: return None matches = validate( @@ -163,9 +161,7 @@ chunk_header, ) # XX FIXME: we discard chunk extensions. Does anyone care? - # We convert to bytes because Python 2's `int()` function doesn't - # work properly on bytearray objects. - self._bytes_in_chunk = int(bytes(matches["chunk_size"]), base=16) + self._bytes_in_chunk = int(matches["chunk_size"], base=16) if self._bytes_in_chunk == 0: self._reading_trailer = True return self(buf) @@ -191,7 +187,7 @@ ) -class Http10Reader(object): +class Http10Reader: def __call__(self, buf): data = buf.maybe_extract_at_most(999999999) if data is None: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/_receivebuffer.py new/h11-0.12.0/h11/_receivebuffer.py --- old/h11-0.11.0/h11/_receivebuffer.py 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/h11/_receivebuffer.py 2021-01-01 12:34:33.000000000 +0100 @@ -1,10 +1,12 @@ +import re import sys __all__ = ["ReceiveBuffer"] # Operations we want to support: -# - find next \r\n or \r\n\r\n, or wait until there is one +# - find next \r\n or \r\n\r\n (\n or \n\n are also acceptable), +# or wait until there is one # - read at-most-N bytes # Goals: # - on average, do this fast @@ -38,75 +40,113 @@ # slightly clever thing where we delay calling compress() until we've # processed a whole event, which could in theory be slightly more efficient # than the internal bytearray support.) -class ReceiveBuffer(object): +blank_line_regex = re.compile(b"\n\r?\n", re.MULTILINE) + + +class ReceiveBuffer: def __init__(self): self._data = bytearray() - # These are both absolute offsets into self._data: - self._start = 0 - self._looked_at = 0 - self._looked_for = b"" + self._next_line_search = 0 + self._multiple_lines_search = 0 + + def __iadd__(self, byteslike): + self._data += byteslike + return self def __bool__(self): return bool(len(self)) + def __len__(self): + return len(self._data) + # for @property unprocessed_data def __bytes__(self): - return bytes(self._data[self._start :]) - - if sys.version_info[0] < 3: # version specific: Python 2 - __str__ = __bytes__ - __nonzero__ = __bool__ + return bytes(self._data) - def __len__(self): - return len(self._data) - self._start + def _extract(self, count): + # extracting an initial slice of the data buffer and return it + out = self._data[:count] + del self._data[:count] - def compress(self): - # Heuristic: only compress if it lets us reduce size by a factor - # of 2 - if self._start > len(self._data) // 2: - del self._data[: self._start] - self._looked_at -= self._start - self._start -= self._start + self._next_line_search = 0 + self._multiple_lines_search = 0 - def __iadd__(self, byteslike): - self._data += byteslike - return self + return out def maybe_extract_at_most(self, count): - out = self._data[self._start : self._start + count] + """ + Extract a fixed number of bytes from the buffer. + """ + out = self._data[:count] if not out: return None - self._start += len(out) - return out - def maybe_extract_until_next(self, needle): - # Returns extracted bytes on success (advancing offset), or None on - # failure - if self._looked_for == needle: - search_start = max(self._start, self._looked_at - len(needle) + 1) - else: - search_start = self._start - offset = self._data.find(needle, search_start) - if offset == -1: - self._looked_at = len(self._data) - self._looked_for = needle + return self._extract(count) + + def maybe_extract_next_line(self): + """ + Extract the first line, if it is completed in the buffer. + """ + # Only search in buffer space that we've not already looked at. + search_start_index = max(0, self._next_line_search - 1) + partial_idx = self._data.find(b"\r\n", search_start_index) + + if partial_idx == -1: + self._next_line_search = len(self._data) return None - new_start = offset + len(needle) - out = self._data[self._start : new_start] - self._start = new_start - return out - # HTTP/1.1 has a number of constructs where you keep reading lines until - # you see a blank one. This does that, and then returns the lines. + # + 2 is to compensate len(b"\r\n") + idx = partial_idx + 2 + + return self._extract(idx) + def maybe_extract_lines(self): - if self._data[self._start : self._start + 2] == b"\r\n": - self._start += 2 + """ + Extract everything up to the first blank line, and return a list of lines. + """ + # Handle the case where we have an immediate empty line. + if self._data[:1] == b"\n": + self._extract(1) + return [] + + if self._data[:2] == b"\r\n": + self._extract(2) return [] - else: - data = self.maybe_extract_until_next(b"\r\n\r\n") - if data is None: - return None - lines = data.split(b"\r\n") - assert lines[-2] == lines[-1] == b"" - del lines[-2:] - return lines + + # Only search in buffer space that we've not already looked at. + match = blank_line_regex.search(self._data, self._multiple_lines_search) + if match is None: + self._multiple_lines_search = max(0, len(self._data) - 2) + return None + + # Truncate the buffer and return it. + idx = match.span(0)[-1] + out = self._extract(idx) + lines = out.split(b"\n") + + for line in lines: + if line.endswith(b"\r"): + del line[-1] + + assert lines[-2] == lines[-1] == b"" + + del lines[-2:] + + return lines + + # In theory we should wait until `\r\n` before starting to validate + # incoming data. However it's interesting to detect (very) invalid data + # early given they might not even contain `\r\n` at all (hence only + # timeout will get rid of them). + # This is not a 100% effective detection but more of a cheap sanity check + # allowing for early abort in some useful cases. + # This is especially interesting when peer is messing up with HTTPS and + # sent us a TLS stream where we were expecting plain HTTP given all + # versions of TLS so far start handshake with a 0x16 message type code. + def is_next_line_obviously_invalid_request_line(self): + try: + # HTTP header line must not contain non-printable characters + # and should not start with a space + return self._data[0] < 0x21 + except IndexError: + return False diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/_state.py new/h11-0.12.0/h11/_state.py --- old/h11-0.11.0/h11/_state.py 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/h11/_state.py 2021-01-01 12:34:33.000000000 +0100 @@ -197,7 +197,7 @@ } -class ConnectionState(object): +class ConnectionState: def __init__(self): # Extra bits of state that don't quite fit into the state model. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/_util.py new/h11-0.12.0/h11/_util.py --- old/h11-0.11.0/h11/_util.py 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/h11/_util.py 2021-01-01 12:34:33.000000000 +0100 @@ -1,6 +1,3 @@ -import re -import sys - __all__ = [ "ProtocolError", "LocalProtocolError", @@ -74,34 +71,17 @@ # (exc_info[0]) separately from the exception object (exc_info[1]), # and we only modified the latter. So we really do need to re-raise # the new type explicitly. - if sys.version_info[0] >= 3: - # On py3, the traceback is part of the exception object, so our - # in-place modification preserved it and we can just re-raise: - raise self - else: - # On py2, preserving the traceback requires 3-argument - # raise... but on py3 this is a syntax error, so we have to hide - # it inside an exec - exec("raise RemoteProtocolError, self, sys.exc_info()[2]") + # On py3, the traceback is part of the exception object, so our + # in-place modification preserved it and we can just re-raise: + raise self class RemoteProtocolError(ProtocolError): pass -try: - _fullmatch = type(re.compile("")).fullmatch -except AttributeError: - - def _fullmatch(regex, data): # version specific: Python < 3.4 - match = regex.match(data) - if match and match.end() != len(data): - match = None - return match - - def validate(regex, data, msg="malformed data", *format_args): - match = _fullmatch(regex, data) + match = regex.fullmatch(data) if not match: if format_args: msg = msg.format(*format_args) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/_version.py new/h11-0.12.0/h11/_version.py --- old/h11-0.11.0/h11/_version.py 2020-10-05 20:23:01.000000000 +0200 +++ new/h11-0.12.0/h11/_version.py 2021-01-01 12:34:33.000000000 +0100 @@ -13,4 +13,4 @@ # want. (Contrast with the special suffix 1.0.0.dev, which sorts *before* # 1.0.0.) -__version__ = "0.11.0" +__version__ = "0.12.0" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/_writers.py new/h11-0.12.0/h11/_writers.py --- old/h11-0.11.0/h11/_writers.py 2020-10-05 20:22:54.000000000 +0200 +++ new/h11-0.12.0/h11/_writers.py 2021-01-01 12:34:33.000000000 +0100 @@ -7,32 +7,12 @@ # - a writer # - or, for body writers, a dict of framin-dependent writer factories -import sys - from ._events import Data, EndOfMessage from ._state import CLIENT, IDLE, SEND_BODY, SEND_RESPONSE, SERVER from ._util import LocalProtocolError __all__ = ["WRITERS"] -# Equivalent of bstr % values, that works on python 3.x for x < 5 -if (3, 0) <= sys.version_info < (3, 5): - - def bytesmod(bstr, values): - decoded_values = [] - for value in values: - if isinstance(value, bytes): - decoded_values.append(value.decode("ascii")) - else: - decoded_values.append(value) - return (bstr.decode("ascii") % tuple(decoded_values)).encode("ascii") - - -else: - - def bytesmod(bstr, values): - return bstr % values - def write_headers(headers, write): # "Since the Host field-value is critical information for handling a @@ -41,17 +21,17 @@ raw_items = headers._full_items for raw_name, name, value in raw_items: if name == b"host": - write(bytesmod(b"%s: %s\r\n", (raw_name, value))) + write(b"%s: %s\r\n" % (raw_name, value)) for raw_name, name, value in raw_items: if name != b"host": - write(bytesmod(b"%s: %s\r\n", (raw_name, value))) + write(b"%s: %s\r\n" % (raw_name, value)) write(b"\r\n") def write_request(request, write): if request.http_version != b"1.1": raise LocalProtocolError("I only send HTTP/1.1") - write(bytesmod(b"%s %s HTTP/1.1\r\n", (request.method, request.target))) + write(b"%s %s HTTP/1.1\r\n" % (request.method, request.target)) write_headers(request.headers, write) @@ -68,11 +48,11 @@ # from stdlib's http.HTTPStatus table. Or maybe just steal their enums # (either by import or copy/paste). We already accept them as status codes # since they're of type IntEnum < int. - write(bytesmod(b"HTTP/1.1 %s %s\r\n", (status_bytes, response.reason))) + write(b"HTTP/1.1 %s %s\r\n" % (status_bytes, response.reason)) write_headers(response.headers, write) -class BodyWriter(object): +class BodyWriter: def __call__(self, event, write): if type(event) is Data: self.send_data(event.data, write) @@ -111,7 +91,7 @@ # end-of-message. if not data: return - write(bytesmod(b"%x\r\n", (len(data),))) + write(b"%x\r\n" % len(data)) write(data) write(b"\r\n") diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/tests/test_against_stdlib_http.py new/h11-0.12.0/h11/tests/test_against_stdlib_http.py --- old/h11-0.11.0/h11/tests/test_against_stdlib_http.py 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/h11/tests/test_against_stdlib_http.py 2021-01-01 12:34:33.000000000 +0100 @@ -1,26 +1,14 @@ import json import os.path import socket +import socketserver import threading from contextlib import closing, contextmanager +from http.server import SimpleHTTPRequestHandler +from urllib.request import urlopen import h11 -try: - from urllib.request import urlopen -except ImportError: # version specific: Python 2 - from urllib2 import urlopen - -try: - import socketserver -except ImportError: # version specific: Python 2 - import SocketServer as socketserver - -try: - from http.server import SimpleHTTPRequestHandler -except ImportError: # version specific: Python 2 - from SimpleHTTPServer import SimpleHTTPRequestHandler - @contextmanager def socket_server(handler): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/tests/test_connection.py new/h11-0.12.0/h11/tests/test_connection.py --- old/h11-0.11.0/h11/tests/test_connection.py 2020-10-05 20:22:54.000000000 +0200 +++ new/h11-0.12.0/h11/tests/test_connection.py 2021-01-01 12:34:33.000000000 +0100 @@ -969,6 +969,38 @@ c.next_event() [email protected]( + "data", + [ + b"\x00", + b"\x20", + b"\x16\x03\x01\x00\xa5", # Typical start of a TLS Client Hello + ], +) +def test_early_detection_of_invalid_request(data): + c = Connection(SERVER) + # Early detection should occur before even receiving a `\r\n` + c.receive_data(data) + with pytest.raises(RemoteProtocolError): + c.next_event() + + [email protected]( + "data", + [ + b"\x00", + b"\x20", + b"\x16\x03\x03\x00\x31", # Typical start of a TLS Server Hello + ], +) +def test_early_detection_of_invalid_response(data): + c = Connection(CLIENT) + # Early detection should occur before even receiving a `\r\n` + c.receive_data(data) + with pytest.raises(RemoteProtocolError): + c.next_event() + + # This used to give different headers for HEAD and GET. # The correct way to handle HEAD is to put whatever headers we *would* have # put if it were a GET -- even though we know that for HEAD, those headers diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/tests/test_events.py new/h11-0.12.0/h11/tests/test_events.py --- old/h11-0.11.0/h11/tests/test_events.py 2020-10-05 20:22:54.000000000 +0200 +++ new/h11-0.12.0/h11/tests/test_events.py 2021-01-01 12:34:33.000000000 +0100 @@ -1,3 +1,5 @@ +from http import HTTPStatus + import pytest from .. import _events @@ -154,10 +156,6 @@ def test_intenum_status_code(): # https://github.com/python-hyper/h11/issues/72 - try: - from http import HTTPStatus - except ImportError: - pytest.skip("Only affects Python 3") r = Response(status_code=HTTPStatus.OK, headers=[], http_version="1.0") assert r.status_code == HTTPStatus.OK diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/tests/test_io.py new/h11-0.12.0/h11/tests/test_io.py --- old/h11-0.11.0/h11/tests/test_io.py 2020-10-05 20:22:54.000000000 +0200 +++ new/h11-0.12.0/h11/tests/test_io.py 2021-01-01 12:34:33.000000000 +0100 @@ -215,6 +215,43 @@ ), ) + # Tolerate headers line endings (\r\n and \n) + # \n\r\b between headers and body + tr( + READERS[SERVER, SEND_RESPONSE], + b"HTTP/1.1 200 OK\r\nSomeHeader: val\n\r\n", + Response( + status_code=200, + headers=[("SomeHeader", "val")], + http_version="1.1", + reason="OK", + ), + ) + + # delimited only with \n + tr( + READERS[SERVER, SEND_RESPONSE], + b"HTTP/1.1 200 OK\nSomeHeader1: val1\nSomeHeader2: val2\n\n", + Response( + status_code=200, + headers=[("SomeHeader1", "val1"), ("SomeHeader2", "val2")], + http_version="1.1", + reason="OK", + ), + ) + + # mixed \r\n and \n + tr( + READERS[SERVER, SEND_RESPONSE], + b"HTTP/1.1 200 OK\r\nSomeHeader1: val1\nSomeHeader2: val2\n\r\n", + Response( + status_code=200, + headers=[("SomeHeader1", "val1"), ("SomeHeader2", "val2")], + http_version="1.1", + reason="OK", + ), + ) + # obsolete line folding tr( READERS[CLIENT, IDLE], diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/tests/test_receivebuffer.py new/h11-0.12.0/h11/tests/test_receivebuffer.py --- old/h11-0.11.0/h11/tests/test_receivebuffer.py 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/h11/tests/test_receivebuffer.py 2021-01-01 12:34:33.000000000 +0100 @@ -1,3 +1,7 @@ +import re + +import pytest + from .._receivebuffer import ReceiveBuffer @@ -12,7 +16,6 @@ assert len(b) == 3 assert bytes(b) == b"123" - b.compress() assert bytes(b) == b"123" assert b.maybe_extract_at_most(2) == b"12" @@ -20,7 +23,6 @@ assert len(b) == 1 assert bytes(b) == b"3" - b.compress() assert bytes(b) == b"3" assert b.maybe_extract_at_most(10) == b"3" @@ -33,32 +35,35 @@ # maybe_extract_until_next ################################################################ - b += b"12345a6789aa" - - assert b.maybe_extract_until_next(b"a") == b"12345a" - assert bytes(b) == b"6789aa" - - assert b.maybe_extract_until_next(b"aaa") is None - assert bytes(b) == b"6789aa" + b += b"123\n456\r\n789\r\n" - b += b"a12" - assert b.maybe_extract_until_next(b"aaa") == b"6789aaa" - assert bytes(b) == b"12" + assert b.maybe_extract_next_line() == b"123\n456\r\n" + assert bytes(b) == b"789\r\n" - # check repeated searches for the same needle, triggering the - # pickup-where-we-left-off logic - b += b"345" - assert b.maybe_extract_until_next(b"aaa") is None + assert b.maybe_extract_next_line() == b"789\r\n" + assert bytes(b) == b"" - b += b"6789aaa123" - assert b.maybe_extract_until_next(b"aaa") == b"123456789aaa" - assert bytes(b) == b"123" + b += b"12\r" + assert b.maybe_extract_next_line() is None + assert bytes(b) == b"12\r" + + b += b"345\n\r" + assert b.maybe_extract_next_line() is None + assert bytes(b) == b"12\r345\n\r" + + # here we stopped at the middle of b"\r\n" delimiter + + b += b"\n6789aaa123\r\n" + assert b.maybe_extract_next_line() == b"12\r345\n\r\n" + assert b.maybe_extract_next_line() == b"6789aaa123\r\n" + assert b.maybe_extract_next_line() is None + assert bytes(b) == b"" ################################################################ # maybe_extract_lines ################################################################ - b += b"\r\na: b\r\nfoo:bar\r\n\r\ntrailing" + b += b"123\r\na: b\r\nfoo:bar\r\n\r\ntrailing" lines = b.maybe_extract_lines() assert lines == [b"123", b"a: b", b"foo:bar"] assert bytes(b) == b"trailing" @@ -76,3 +81,54 @@ b += b"\r\ntrailing" assert b.maybe_extract_lines() == [] assert bytes(b) == b"trailing" + + [email protected]( + "data", + [ + pytest.param( + ( + b"HTTP/1.1 200 OK\r\n", + b"Content-type: text/plain\r\n", + b"Connection: close\r\n", + b"\r\n", + b"Some body", + ), + id="with_crlf_delimiter", + ), + pytest.param( + ( + b"HTTP/1.1 200 OK\n", + b"Content-type: text/plain\n", + b"Connection: close\n", + b"\n", + b"Some body", + ), + id="with_lf_only_delimiter", + ), + pytest.param( + ( + b"HTTP/1.1 200 OK\n", + b"Content-type: text/plain\r\n", + b"Connection: close\n", + b"\n", + b"Some body", + ), + id="with_mixed_crlf_and_lf", + ), + ], +) +def test_receivebuffer_for_invalid_delimiter(data): + b = ReceiveBuffer() + + for line in data: + b += line + + lines = b.maybe_extract_lines() + + assert lines == [ + b"HTTP/1.1 200 OK", + b"Content-type: text/plain", + b"Connection: close", + ] + assert bytes(b) == b"Some body" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11/tests/test_util.py new/h11-0.12.0/h11/tests/test_util.py --- old/h11-0.11.0/h11/tests/test_util.py 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/h11/tests/test_util.py 2021-01-01 12:34:33.000000000 +0100 @@ -93,7 +93,7 @@ assert bytesify("123") == b"123" with pytest.raises(UnicodeEncodeError): - bytesify(u"\u1234") + bytesify("\u1234") with pytest.raises(TypeError): bytesify(10) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/h11.egg-info/PKG-INFO new/h11-0.12.0/h11.egg-info/PKG-INFO --- old/h11-0.11.0/h11.egg-info/PKG-INFO 2020-10-05 20:23:10.000000000 +0200 +++ new/h11-0.12.0/h11.egg-info/PKG-INFO 2021-01-01 12:34:42.000000000 +0100 @@ -1,6 +1,6 @@ -Metadata-Version: 1.1 +Metadata-Version: 1.2 Name: h11 -Version: 0.11.0 +Version: 0.12.0 Summary: A pure-Python, bring-your-own-I/O implementation of HTTP/1.1 Home-page: https://github.com/python-hyper/h11 Author: Nathaniel J. Smith @@ -120,7 +120,8 @@ It has a test suite with 100.0% coverage for both statements and branches. - Currently it supports Python 3 (testing on 3.5-3.8), Python 2.7, and PyPy. + Currently it supports Python 3 (testing on 3.6-3.9) and PyPy 3. + The last Python 2-compatible version was h11 0.11.x. (Originally it had a Cython wrapper for `http-parser <https://github.com/nodejs/http-parser>`_ and a beautiful nested state machine implemented with ``yield from`` to postprocess the output. But @@ -180,12 +181,12 @@ Classifier: License :: OSI Approved :: MIT License Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy -Classifier: Programming Language :: Python :: 2 -Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 -Classifier: Programming Language :: Python :: 3.5 +Classifier: Programming Language :: Python :: 3 :: Only Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: 3.8 +Classifier: Programming Language :: Python :: 3.9 Classifier: Topic :: Internet :: WWW/HTTP Classifier: Topic :: System :: Networking +Requires-Python: >=3.6 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/setup.cfg new/h11-0.12.0/setup.cfg --- old/h11-0.11.0/setup.cfg 2020-10-05 20:23:10.682039700 +0200 +++ new/h11-0.12.0/setup.cfg 2021-01-01 12:34:42.171725700 +0100 @@ -1,6 +1,3 @@ -[bdist_wheel] -universal = 1 - [isort] combine_as_imports = True force_grid_wrap = 0 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/h11-0.11.0/setup.py new/h11-0.12.0/setup.py --- old/h11-0.11.0/setup.py 2020-08-13 11:49:16.000000000 +0200 +++ new/h11-0.12.0/setup.py 2021-01-01 12:34:33.000000000 +0100 @@ -17,19 +17,19 @@ # This means, just install *everything* you see under h11/, even if it # doesn't look like a source file, so long as it appears in MANIFEST.in: include_package_data=True, + python_requires=">=3.6", classifiers=[ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", - "Programming Language :: Python :: 2", - "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", - "Programming Language :: Python :: 3.5", + "Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", "Topic :: Internet :: WWW/HTTP", "Topic :: System :: Networking", ],
