[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Changes by Serhiy Storchaka storch...@gmail.com: -- assignee: - serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Nick Coghlan added the comment: Patch looks good to me, but if any specific features are needed to work around misbehaving codecs (as per issue 20132), a comment in the appropriate place referencing that issue would be helpful. And if that workaround means we can remove the special casing from the test_readlines test for the binary transform, cool :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Serhiy Storchaka added the comment: Actually this patch doesn't work around misbehaving codecs. It just makes specific tests (one readline, one read) be passed. More complex tests which use multiple readline's or read's still can fail with these misbehaving codecs. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Roundup Robot added the comment: New changeset e24265eb2271 by Serhiy Storchaka in branch '2.7': Issue #8260: The read(), readline() and readlines() methods of http://hg.python.org/cpython/rev/e24265eb2271 New changeset 9c96c266896e by Serhiy Storchaka in branch '3.3': Issue #8260: The read(), readline() and readlines() methods of http://hg.python.org/cpython/rev/9c96c266896e New changeset b72508a785de by Serhiy Storchaka in branch 'default': Issue #8260: The read(), readline() and readlines() methods of http://hg.python.org/cpython/rev/b72508a785de -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Changes by Serhiy Storchaka storch...@gmail.com: -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Serhiy Storchaka added the comment: Here is revised patch. * Behavior is changed less. read() is less greedy and uses characters from the buffer when read() is called with only one argument (size). It is now a little closer to io stream's read() than with previous patch. * Added tests for cases of issue12446 and issue16636. * Fixed read() for for the TransformCodecTest.test_read test added in 3.4. Actually the uu_codec and zlib_codec are broken. -- versions: -Python 3.2 Added file: http://bugs.python.org/file33410/codecs_read-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Serhiy Storchaka added the comment: I think the patch is wrong or is not optimal for case when chars is -1, but size is not. If we want to read all data in any case, then we should call self.stream.read() without argument if chars 0 or size 0. If we want to read no more than size bytes, then all loop code should be totally rewritten. Perhaps I am wrong. -- nosy: +serhiy.storchaka versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Serhiy Storchaka added the comment: As showed in issue12446, issue14475 and issue16636 there are different methods to reproduce this bug (read(size, chars) + readlines(), readline() + readlines()). All this cases should be tested. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
STINNER Victor victor.stin...@haypocalc.com added the comment: See also #12446. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
harobed steph...@harobed.org added the comment: Up, I think this patch isn't applied in Python 3.3a0. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Changes by STINNER Victor victor.stin...@haypocalc.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
R. David Murray rdmur...@bitdance.com added the comment: According to this ticket it hasn't been applied anywhere yet (a message will be posted here when it is). -- nosy: +r.david.murray versions: +Python 3.3 -Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Éric Araujo mer...@netwok.org added the comment: I applied the diff to test_codecs in py3k, removed the u prefixes and ran: failure. I applied the fix and the test passed. -- nosy: +eric.araujo title: When I use codecs.open(...) and f.readline() follow up by f.read() return bad result - When I use codecs.open(...) and f.readline() follow up by f.read() return bad result versions: +Python 2.7, Python 3.1, Python 3.2 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Daniel Diniz aja...@gmail.com added the comment: Hi Stephane, I think you're seeing different buffering behavior, which I suspect is correct according to docs. codecs.open should default to line buffering[1], while open uses the system default[2]. The read() where the assert fails is returning the remaining buffer from the readline (which read 72 chars). Asserting e.g. f.read(1024) == ... will give you the expected result. [1] http://docs.python.org/library/codecs.html#codecs.open [2] http://docs.python.org/library/functions.html#open -- nosy: +ajaksu2 priority: - normal stage: - test needed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Buffering applies when writing, not when reading a file. There is indeed a problem in codecs.py: after a readline(), read() will return the content of the internal buffer, and not more. The size parameter is a hint, and should not be used to decide whether the character buffer is enough to satisfy the read() request. Patch is attached, with test. -- keywords: +patch nosy: +amaury.forgeotdarc stage: test needed - patch review Added file: http://bugs.python.org/file16705/codecs_read.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Marc-Andre Lemburg m...@egenix.com added the comment: Amaury Forgeot d'Arc wrote: Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Buffering applies when writing, not when reading a file. There is indeed a problem in codecs.py: after a readline(), read() will return the content of the internal buffer, and not more. The size parameter is a hint, and should not be used to decide whether the character buffer is enough to satisfy the read() request. Patch is attached, with test. Agreed. The patch looks good except the if-line should read: if chars = 0 and len(self.charbuffer) = chars: ... Thanks, -- Marc-Andre Lemburg eGenix.com ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ -- nosy: +lemburg title: When I use codecs.open(...) and f.readline() follow up by f.read() return bad result - When I use codecs.open(...) and f.readline() follow up by f.read() return bad result ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Updated patch. [I also tried to avoid reading the underlying file if len(self.bytebuffer)=size, but it does not work with multibytes chars when size=1] -- Added file: http://bugs.python.org/file16706/codecs_read-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8260] When I use codecs.open(...) and f.readline() follow up by f.read() return bad result
New submission from harobed steph...@harobed.org:
This is an example, last assert return an error :
f = open('data.txt', 'w')
f.write(line 1
line 2
line 3
line 4
line 5
line 6
line 7
line 8
line 9
line 10
line 11
)
f.close()
f = open('data.txt', 'r')
assert f.readline() == 'line 1\n'
assert f.read() == line 2
line 3
line 4
line 5
line 6
line 7
line 8
line 9
line 10
line 11
f.close()
import codecs
f = codecs.open('data.txt', 'r', 'utf8')
assert f.read() == line 1
line 2
line 3
line 4
line 5
line 6
line 7
line 8
line 9
line 10
line 11
f.close()
f = codecs.open('data.txt', 'r', 'utf8')
assert f.readline() == 'line 1\n'
# this assert return a ERROR
assert f.read() == line 2
line 3
line 4
line 5
line 6
line 7
line 8
line 9
line 10
line 11
f.close()
Regards,
Stephane
--
components: Library (Lib)
messages: 101892
nosy: harobed
severity: normal
status: open
title: When I use codecs.open(...) and f.readline() follow up by f.read()
return bad result
type: behavior
versions: Python 2.6
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8260
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
