[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Roundup Robot added the comment: New changeset 5ca6e8af0aab by Nadeem Vawda in branch '3.3': #18430: Document that peek() may change the position of the underlying file for http://hg.python.org/cpython/rev/5ca6e8af0aab New changeset 0f587fe304be by Nadeem Vawda in branch 'default': Closes #18430: Document that peek() may change the position of the underlying http://hg.python.org/cpython/rev/0f587fe304be -- nosy: +python-dev resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: -serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
R. David Murray added the comment: I don't think the comment about save and restore of the file position adds enough value to be worth the possible confusion (do I only need to save and restore around peek? Why?) I think the sentence should read, ex, Calling a :class:`LZMAFile` object's :meth:`peek` method does not advance its access position, but the access position of the underlying :term:`file object` may be affected. Note that I have no objection to making the two sync up, if it is in fact possible. But if Serhiy says it can't be, I expect he's right. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Serhiy Storchaka added the comment: A file can be non-seekable (pipe, socket) and we can't unread unused data back. We can't implement GzipFile.peek() in terms of underlied file's peek() because peek() doesn't guaranteed return more than one byte, but it should return at least one byte if the file is not ended. We can't detect the end of compressed data without reading some data past the end of compressed data. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Nick Bargnesi added the comment: I'd suggest sticking with file position instead of switching to access position. E.g., the complete lzma wording would be: Return buffered data without advancing the file position. At least one byte of data will be returned, unless EOF has been reached. The exact number of bytes returned is unspecified (the *size* argument is ignored). Although calling a :class:`LZMAFile` object's :meth:`peek` method does not advance its file position, the file position of the underlying :term:`file object` may be affected. The point about mentioning save and restore notwithstanding, *any* documentation about the effect on position change is a step in the right direction. If the file position not changing is the better scenario, having the side effect documented is at least good. We can save people's time by being explicit about the side effect _and_ maintain code readability. Imagine hunting for a bug manifesting itself as a change in file position only to find out a peek() call was the cause. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
R. David Murray added the comment: I chose 'access position' because that's the terminology used in the 'seek' man page. 'file position' sounds odd to me, but since that is the term already used it does make sense to be consistent. -- assignee: - docs@python components: +Documentation -Library (Lib) nosy: +docs@python stage: - needs patch versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
New submission from Nick Bargnesi:
Using existing file objects as arguments to the open functions in the gzip,
bz2, and lzma libraries can cause the underlying fileobj position to get
changed - and not quite in ways one would expect.
Calling peek against the returned file objects -- gzip.GzipFile, bz2.BZ2File,
and lzma.LZMAFile will in one scenario advance the position of the supplied
file object:
import bz2
fileobj = open('test.bz2', mode='rb')
bzfile = bz2.open(fileobj)
# file positions at 0
assert fileobj.tell() == 0, bzfile.tell() == 0
bzfile.peek()
b'Test file.\n'
fileobj.tell()
52
If after the initial peek, we rewind the underlying fileobj and peek again, the
behavior changes:
fileobj.seek(0)
0
bzfile.peek()
b'Test file.\n'
fileobj.tell()
0
The second scenario serves to complicate things a bit with the change in
behavior.
I would be less surprised if the module documentation simply stated the affect
on file position of the file object being used. Though it would be beautiful if
the underlying file object didn't change at all. The latter seems possible
since the three modules know whether the file object is seekable.
The gzip and lzma modules exhibit similar behavior - gzip for example:
import gzip
fileobj = open('test.gz', mode='rb')
gzfile = gzip.open(fileobj)
# file positions at 0
assert fileobj.tell() == 0 and gzfile.tell() == 0
gzfile.peek(1)
b'Test file.\n'
fileobj.tell()
36
# rewind, and do it again
fileobj.seek(0)
gzfile.peek(1)
b'Test file.\n'
fileobj.tell()
0
--
components: Library (Lib)
messages: 192890
nosy: nbargnesi
priority: normal
severity: normal
status: open
title: gzip, bz2, lzma: peek advances file position of existing file object
type: enhancement
versions: Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18430
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Serhiy Storchaka added the comment: I guess this issue can't be fixed. -- nosy: +nadeem.vawda, serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Madison May added the comment: Why would something like the following work? #At the beginning of peek() #keep track of prior offset position = self.fileobj.tell() #immediately before return statement #restore previous fileobj offset self.fileobj.seek(position) -- nosy: +madison.may ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Madison May added the comment: *wouldn't -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
R. David Murray added the comment: Regardless of whether or not it *can* be fixed, I personally would not expect the file position to be either unchanged or predictable. The file object is being passed to something that is going to read and/or write from it, after all. If the calling application needs the file to be in a particular seek position, it should save it and reset it, I think. -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Nick Bargnesi added the comment: In that the underlying file position is not deterministic when used like this, I'm inclined to agree. Though faced with documentation stating it does not advance the file position, it certainly is less than explicit about it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
R. David Murray added the comment: Hmm, yes. Perhaps it should say does not advance the gzip read position? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Nick Bargnesi added the comment: That's an improvement. Using wording similar to the constructor's: Calling a GzipFile object’s peek() method does not advance its position, but the fileobj may be affected. The caller may wish to save the fileobj position prior to calling peek() and resetting it upon return. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Madison May added the comment: Sounds good to me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18430] gzip, bz2, lzma: peek advances file position of existing file object
Nick Bargnesi added the comment: Proposed documentation change to gzip, bz2, and lzma modules, in patch form. -- keywords: +patch Added file: http://bugs.python.org/file30896/issue18430.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
