[issue43260] Never release buffer when MemoryError in print()

STINNER Victor added the comment: I didn't understand the bug but thanks for fixing it :) -- ___ Python tracker ___ ___

[issue43260] Never release buffer when MemoryError in print()

Change by Inada Naoki : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue43260] Never release buffer when MemoryError in print()

Inada Naoki added the comment: New changeset 6e2f144f53982d7103d4cfc2d9361fc445a1817e by Inada Naoki in branch '3.8': bpo-43260: io: Prevent large data remains in textio buffer. (GH-24592) https://github.com/python/cpython/commit/6e2f144f53982d7103d4cfc2d9361fc445a1817e --

[issue43260] Never release buffer when MemoryError in print()

Inada Naoki added the comment: New changeset d51436f95bf5978f82d917e53e9a456fdaa83a9d by Inada Naoki in branch '3.9': bpo-43260: io: Prevent large data remains in textio buffer. (GH-24592) https://github.com/python/cpython/commit/d51436f95bf5978f82d917e53e9a456fdaa83a9d --

[issue43260] Never release buffer when MemoryError in print()

Change by Inada Naoki : -- pull_requests: +23399 pull_request: https://github.com/python/cpython/pull/24618 ___ Python tracker ___

[issue43260] Never release buffer when MemoryError in print()

Change by Inada Naoki : -- pull_requests: +23398 pull_request: https://github.com/python/cpython/pull/24617 ___ Python tracker ___

[issue43260] Never release buffer when MemoryError in print()

Inada Naoki added the comment: New changeset 01806d5beba3d208bb56adba6829097d803bf54f by Inada Naoki in branch 'master': bpo-43260: io: Prevent large data remains in textio buffer. (GH-24592) https://github.com/python/cpython/commit/01806d5beba3d208bb56adba6829097d803bf54f --

[issue43260] Never release buffer when MemoryError in print()

Change by Ramin Farajpour Cami : -- nosy: +vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue43260] Never release buffer when MemoryError in print()

Eryk Sun added the comment: > Isn't `PyUnicode_GET_LENGTH(text) < self->chunk_size` enough? Yes, that's simpler, except with `<=`" instead of `<`, since the maximum count is chunk_size when pending_bytes is a list or ASCII string. When I wrote the more complex check, I did't take into

[issue43260] Never release buffer when MemoryError in print()

Inada Naoki added the comment: You are right. I misunderstood. ``` if (PyUnicode_IS_ASCII(text) && (PyUnicode_GET_LENGTH(text) + (self->pending_bytes ? self->pending_bytes_count : 0)) <= self->chunk_size &&

[issue43260] Never release buffer when MemoryError in print()

Eryk Sun added the comment: > In your code, huge data passed to .write(huge) may be > remained in the internal buffer. If you mean the buffered writer, then I don't see the problem. A large bytes object in pending_bytes gets temporarily referenced by _textiowrapper_writeflush(), and

[issue43260] Never release buffer when MemoryError in print()

Inada Naoki added the comment: In your code, huge data passed to .write(huge) may be remained in the internal buffer. ``` [NEW PRE-FLUSH] else if ((self->pending_bytes_count + bytes_len) > self->chunk_size) { if (_textiowrapper_writeflush(self) < 0) { Py_DECREF(b);

[issue43260] Never release buffer when MemoryError in print()

Eryk Sun added the comment: > stdout.write("small text") > stdout.write("very large text") # Calls writeflush, but can not allocate > buffer. Without the optimization, in most cases this will likely fail in _io_TextIOWrapper_write_impl() at the line `b = (*self->encodefunc)((PyObject *)

[issue43260] Never release buffer when MemoryError in print()

Change by Inada Naoki : -- keywords: +patch pull_requests: +23372 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24592 ___ Python tracker ___

[issue43260] Never release buffer when MemoryError in print()

Inada Naoki added the comment: This is not the problem only in the optimization. _textiowrapper_writeflush need to create buffer bytes object anyway and if it cause MemoryError, the TextIOWrapper can not flush internal buffer forever. stdout.write("small text") stdout.write("very large

[issue43260] Never release buffer when MemoryError in print()

Eryk Sun added the comment: Issue 36748 added the ASCII optimization in write(), so I'm adding Inada Naoki to the nosy list. -- nosy: +methane ___ Python tracker ___

[issue43260] Never release buffer when MemoryError in print()

Eryk Sun added the comment: The sys.stdout TextIOWrapper is stuck in a bad state. When the write() method is called with an ASCII string, it implements an optimization that stores a reference to the str() object in the internal pending_bytes instead of immediately encoding the string.

[issue43260] Never release buffer when MemoryError in print()

Change by Ramin Farajpour Cami : -- type: -> crash ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue43260] Never release buffer when MemoryError in print()

Ramin Farajpour Cami added the comment: Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 16:33:24) [MSC v.1928 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> print("a"*10) Traceback (most recent call last): File "", line 1, in

[issue43260] Never release buffer when MemoryError in print()

Ramin Farajpour Cami added the comment: Version : Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:43:08) [MSC v.1926 32 bit (Intel)] on win32 -- ___ Python tracker ___

[issue43260] Never release buffer when MemoryError in print()

Change by Ramin Farajpour Cami : -- title: Never release buffer when MemoryError in prtin() -> Never release buffer when MemoryError in print() ___ Python tracker ___