[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Éric Araujo added the comment: As a new feature, this can’t go into older versions. -- ___ Python tracker ___ ___ Python-bugs-list mai

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Oliver Deppert added the comment: Hi, thanks for the patch. Could you also publish a version for older python 2.x ? regards, Olli -- nosy: +Kontr-Olli ___ Python tracker ___ __

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Antoine Pitrou added the comment: Thank you very much, Nadeem. The patch is now in. -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker _

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Roundup Robot added the comment: New changeset 2cb07a46f4b5 by Antoine Pitrou in branch 'default': Issue #5863: Rewrite BZ2File in pure Python, and allow it to accept http://hg.python.org/cpython/rev/2cb07a46f4b5 -- nosy: +python-dev ___ Python track

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: ... and the corresponding updated documentation patch. -- Added file: http://bugs.python.org/file21508/bz2-v6-doc.diff ___ Python tracker ___

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: Here's the updated patch. -- Added file: http://bugs.python.org/file21507/bz2-v6.diff ___ Python tracker ___ _

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Antoine Pitrou added the comment: > The tests for readline() and readlines() expect a TypeError if size is None. > Calling size.__index__() in this case raises an AttributeError instead. > Should I > change the tests to expect an AttributeError? Alternatively, something like > this > would mor

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: Thanks for the review. I've made most of the changes you suggested, but there's one thing I wanted to check about: > - instead of "int(size)", use "size = size.__index__()" so as to forbid floats The tests for readline() and readlines() expect a TypeError if siz

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Antoine Pitrou added the comment: > Here is an updated patch that adds read1() to BZ2File. This should fix things > for issue10791 from the bz2 side. I also took the opportunity to clean up > _read_block() to be more readable. As per Martin's suggestion on python-dev, I > put the copyright notic

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: Updated documentation. -- Added file: http://bugs.python.org/file21503/bz2-v5-doc.diff ___ Python tracker ___

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: Here is an updated patch that adds read1() to BZ2File. This should fix things for issue10791 from the bz2 side. I also took the opportunity to clean up _read_block() to be more readable. As per Martin's suggestion on python-dev, I put the copyright notice in the p

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: > Apparently the PSF has received your contributor agreement. Great; I was just about to send them an email to check. > Does it mean the situation is cleared? I plan to do a review of your latest > patch. Yes, everything's sorted out. Go ahead :) --

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Antoine Pitrou added the comment: Nadeem, > Can I ask that you not commit this > patch until the CA has been submitted? I will need to clear it with my > employer, and it might complicate things if the code in question has > already been committed. Apparently the PSF has received your contribu

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Changes by Andrew Svetlov : -- nosy: +asvetlov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Antoine Pitrou added the comment: > >From the discussion on python-dev, it seems that I will need to submit a > Contributor Agreement to the PSF. Can I ask that you not commit this > patch until the CA has been submitted? I will need to clear it with my > employer, and it might complicate things

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: >From the discussion on python-dev, it seems that I will need to submit a Contributor Agreement to the PSF. Can I ask that you not commit this patch until the CA has been submitted? I will need to clear it with my employer, and it might complicate things if the co

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: Corresponding patch for the module docs. -- Added file: http://bugs.python.org/file21315/bz2-v4-doc.diff ___ Python tracker ___ __

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: Here is an updated patch, incorporating the feedback from your review. The new patch no longer checks for errors in bz2CompressEnd()/bz2DecompressEnd() in the dealloc functions for BZ2Compressor/BZ2Decompressor. I found that calling PyErr_WriteUnraisable() result

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Antoine Pitrou added the comment: Given the absence of response on python-dev, I'd say simply remove the obsolete copyright notice. -- ___ Python tracker ___ ___

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: > Well, I would personally advocate not re-adding a license boilerplate, > since it doesn't serve a purpose (nearly all of Python is freely usable > under the PSF License, and the authors are documented by version control). That sounds sensible to me. I'll see wh

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Antoine Pitrou added the comment: > Regarding the use of PY_SSIZE_T_CLEAN, I assume that Py_ssize_t is to be > preferred over plain ssize_t. Is this correct? Yes, ssize_t doesn't exist everywhere AFAIK. (size_t does, or at least we assume it does) > Also, I was wondering whether I need to add

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: > Would it be possible to add an open() function to the bz2 module? Yes, it would be quite trivial, though I don't think it would be worthwhile - all it would do is provide a direct alias for the BZ2File constructor. But as Antoine said, that is a topic for a sep

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Antoine Pitrou added the comment: > Would it be possible to add an open() function to the bz2 module? > Currently gzip has such a function, but bz2 does not: Well, it could be a topic for a separate issue. -- ___ Python tracker

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Michiel de Hoon added the comment: Would it be possible to add an open() function to the bz2 module? Currently gzip has such a function, but bz2 does not: >>> import gzip >>> gzip.open >>> import bz2 >>> bz2.open Traceback (most recent call last): File "", line 1, in ? AttributeError: 'modu

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Nadeem Vawda added the comment: Thanks for the review. I'll try and have an updated patch ready by next weekend. Regarding your comments: > Is there any reason it doesn't inherit io.BufferedIOBase? No, there isn't; I'll fix that in my revised patch. > Since this is a new start, perhaps we sho

[issue5863] bz2.BZ2File should accept other file-like objects. (issue4274045)

Antoine Pitrou added the comment: Reviewers: nadeem vawda , http://codereview.appspot.com/4274045/diff/1/Lib/bz2.py File Lib/bz2.py (right): http://codereview.appspot.com/4274045/diff/1/Lib/bz2.py#newcode25 Lib/bz2.py:25: class BZ2File: Is there any reason it doesn't inherit io.BufferedIOBase?

[issue5863] bz2.BZ2File should accept other file-like objects.

Antoine Pitrou added the comment: Review posted at http://codereview.appspot.com/4274045/ -- ___ Python tracker ___ ___ Python-bugs-li

[issue5863] bz2.BZ2File should accept other file-like objects.

Antoine Pitrou added the comment: Patch posted for review at http://codereview.appspot.com/4274045/. Still have to do a review though :) -- ___ Python tracker ___ __

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: Weird, the patch didn't upload... -- Added file: http://bugs.python.org/file20721/bz2-v3b.diff ___ Python tracker ___

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: Here's a revised version of bz2-v3.diff, with docstrings that are more consistent with the updated documentation. -- ___ Python tracker ___ _

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: Here's an update to the documentation for the bz2 module. -- Added file: http://bugs.python.org/file20692/bz2-doc.diff ___ Python tracker ___

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: OK, I've rewritten the whole bz2 module (patch attached), and I think it is now ready for review. The BZ2File implementation is a cleaned-up version of the one from my previous patch, with some further additions. I've factored out the common compressor/decompre

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: >> * The read*() methods are implemented very inefficiently. Since they >> have to deal with the bytes objects returned by >> BZ2Decompressor.decompress(), a large read results in lots of >> allocations that weren't necessary in the C implementation. > > It probab

[issue5863] bz2.BZ2File should accept other file-like objects.

Antoine Pitrou added the comment: > * The read*() methods are implemented very inefficiently. Since they > have to deal with the bytes objects returned by > BZ2Decompressor.decompress(), a large read results in lots of > allocations that weren't necessary in the C implementation. It probably de

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: Here is a quick-and-dirty reimplementation of BZ2File in Python, on top of the existing C implementation of BZ2Compressor and BZ2Decompressor. There are a couple of issues with this code that need to be fixed: * BZ2Decompressor doesn't signal when it reaches the

[issue5863] bz2.BZ2File should accept other file-like objects.

Antoine Pitrou added the comment: > How would I go about doing this? Would it be sufficient to raise a > DeprecationWarning if the argument is provided by the caller, and add > a note to the docstring and documentation? Yes, totally. -- ___ Python t

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: * I had initially considered doing something *like* that -- ___ Python tracker ___ ___ Python-bugs-lis

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: > Interesting! If you are motivated, a further approach would be to expose > the compressor and decompressor objects from the C extension, and write > the file object in Python (as in Lib/gzip.py). I had initially considered doing something that, but I decided not

[issue5863] bz2.BZ2File should accept other file-like objects.

Antoine Pitrou added the comment: Interesting! If you are motivated, a further approach would be to expose the compressor and decompressor objects from the C extension, and write the file object in Python (as in Lib/gzip.py). > One thing I was unsure of is how to handle exceptions that occur

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: Yes, see bz2module-v1.diff. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue5863] bz2.BZ2File should accept other file-like objects.

Anthony Long added the comment: Are there tests for this? -- nosy: +antlong ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: Here is a patch that rewrites BZ2File to implement the requested feature, and adds some tests using BytesIO objects. Some notes: * iteration and the read*() method now use the same buffering machinery, so they can be mixed freely. The test for issue8397 has bee

[issue5863] bz2.BZ2File should accept other file-like objects.

Nadeem Vawda added the comment: I have been working on a patch for this issue. I've implemented everything except for readline(), readlines() and the iterator protocol. In the existing implementation, the reading methods seem to interact weirdly - iternext() uses a readahead buffer, while non

[issue5863] bz2.BZ2File should accept other file-like objects.

wrobell added the comment: OK! :) -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mai

[issue5863] bz2.BZ2File should accept other file-like objects.

Éric Araujo added the comment: We’ve already agreed the feature is desirable; what’s missing is a patch, not user stories :) -- nosy: +niemeyer versions: +Python 3.3 -Python 3.2 ___ Python tracker

[issue5863] bz2.BZ2File should accept other file-like objects.

wrobell added the comment: A use case wget -O http://planet.openstreetmap.org/planet-110112.osm.bz2 | tee planet.bz2 | osm2sql | psql osm planet-*osm.bz2 files are 14GB at the moment. it would be great to read them from stdin while downloading from a server and uploading to a database at

[issue5863] bz2.BZ2File should accept other file-like objects.

Antoine Pitrou added the comment: > Sorry, I'm giving up. Indeed, I think only an extensive rewrite could fulfill the feature request here. > The copyright notice for bz2module.c lists "Gustavo Niemeyer" as one > of the holders, is he the maintainer? Maybe he should be notified of > this bug.

[issue5863] bz2.BZ2File should accept other file-like objects.

Xuanji Li added the comment: Sorry, I'm giving up. The copyright notice for bz2module.c lists "Gustavo Niemeyer" as one of the holders, is he the maintainer? Maybe he should be notified of this bug. -- ___ Python tracker

[issue5863] bz2.BZ2File should accept other file-like objects.

Xuanji Li added the comment: I'll try working on a patch. -- nosy: +xuanji ___ Python tracker ___ ___ Python-bugs-list mailing list Un

[issue5863] bz2.BZ2File should accept other file-like objects.

Changes by Nadeem Vawda : -- nosy: +nvawda ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.or

[issue5863] bz2.BZ2File should accept other file-like objects.

Raymond Hettinger added the comment: Without a patch and compelling use cases, this has no chance. Recommend closing. -- nosy: +rhettinger ___ Python tracker ___ __

[issue5863] bz2.BZ2File should accept other file-like objects.

Antoine Pitrou added the comment: For the record, this will need a comprehensive rewrite of bz2module, since it uses FILE pointers right now. -- nosy: +pitrou ___ Python tracker ___

[issue5863] bz2.BZ2File should accept other file-like objects.

Éric Araujo added the comment: That’s a perfectly fine reply. Someone will see this feature request and propose a patch eventually. Another way to help is to write tests, since those are in Python. -- ___ Python tracker

[issue5863] bz2.BZ2File should accept other file-like objects.

MizardX added the comment: Would if I could. But, No. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:/

[issue5863] bz2.BZ2File should accept other file-like objects.

Éric Araujo added the comment: Do you want to work on patch? -- components: +Extension Modules -IO, Library (Lib) nosy: +eric.araujo stage: unit test needed -> needs patch ___ Python tracker ___

[issue5863] bz2.BZ2File should accept other file-like objects.

Changes by Mark Lawrence : -- stage: -> unit test needed versions: +Python 3.2 ___ Python tracker ___ ___ Python-bugs-list mailing lis

[issue5863] bz2.BZ2File should accept other file-like objects.

New submission from MizardX : bz2.BZ2File should, like gzip.GzipFile, accept a fileobj argument. If implemented, you could much more easily pipe BZ2-data from other sources, such as stdin or a socket. -- components: IO, Library (Lib) messages: 86716 nosy: MizardX severity: normal statu