[issue15955] gzip, bz2, lzma: add option to limit output size

2014-06-15 Thread Nadeem Vawda
Nadeem Vawda added the comment: Sorry, I just haven't had any free time lately, and may still not be able to give this the attention it deserves for another couple of weeks. Serhiy, would you be interested in reviewing Nikolaus' patch? -- ___ Python

[issue15955] gzip, bz2, lzma: add option to limit output size

2014-04-06 Thread Nadeem Vawda
Nadeem Vawda added the comment: I've posted a review at http://bugs.python.org/review/15955/. (For some reason, it looks like Rietveld didn't send out email notifications. But maybe it never sends a notification to the sender? Hmm.) -- ___ Python

[issue15955] gzip, bz2, lzma: add option to limit output size

2014-03-30 Thread Nadeem Vawda
Nadeem Vawda added the comment: Thanks for the patch, Nikolaus. I'm afraid I haven't had a chance to look over it yet; this past week has been a bit crazy for me. I'll definitely get back to you with a review in the next week, though. -- ___ Python

[issue20781] BZ2File doesn't decompress some .bz2 files correctly

2014-02-27 Thread Nadeem Vawda
Nadeem Vawda added the comment: How does one create a multi-stream bzip2 file in the first place? If you didn't do so deliberately, I would guess that you used a parallel compression tool like pbzip2 or lbzip2 to create your bz2 file. These tools work by splitting the input into chunks

[issue20781] BZ2File doesn't decompress some .bz2 files correctly

2014-02-26 Thread Nadeem Vawda
Nadeem Vawda added the comment: As Serhiy said, multi-stream support was only added to the bz2 module in 3.3, and there is no plan to backport functionality this to 2.7. However, the bz2file package on PyPI [1] does support multi-stream inputs, and you can use its BZ2File class as a drop

[issue15955] gzip, bz2, lzma: add option to limit output size

2014-02-02 Thread Nadeem Vawda
Nadeem Vawda added the comment: After some consideration, I've come to agree with Serhiy that it would be better to keep a private internal buffer, rather than having the user manage unconsumed input data. I'm also in favor of having a flag to indicate whether the decompressor needs more input

[issue20193] Derby: Convert the zlib, _bz2 and _lzma modules to use Argument Clinic

2014-01-26 Thread Nadeem Vawda
Nadeem Vawda added the comment: The latest patch for zlib seems to be missing Modules/zlibmodule.clinic.c I suppose that zdict=b'' have same effect as not specifying zdict. Am I right? Probably, but to be on the safe side I'd prefer that we preserve the behavior of not calling

[issue20193] Derby: Convert the zlib, _bz2 and _lzma modules to use Argument Clinic

2014-01-26 Thread Nadeem Vawda
Nadeem Vawda added the comment: The patch for zlib looks good to me. Thanks for working on this, Serhiy. We're not allowing changes in semantics for Argument Clinic conversion for 3.4. If it doesn't currently accept None, we can't add it right now, and we'll have to save it for 3.5. Fair

[issue20358] test_curses is failing

2014-01-23 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20358 ___ ___ Python-bugs

[issue20358] test_curses is failing

2014-01-23 Thread Nadeem Vawda
Nadeem Vawda added the comment: I can reproduce this (also on Ubuntu 13.10 64-bit). Maybe there's a bug in the version of curses distributed with the latest Ubuntu release? It looks like our only Ubuntu buildbot is using 8.04 (almost 6 years old!). Also note that you won't be able to reproduce

[issue15955] gzip, bz2, lzma: add option to limit output size

2014-01-22 Thread Nadeem Vawda
Nadeem Vawda added the comment: No, I'm afraid I haven't had a chance to do any work on this issue since my last message. I would be happy to review a patch for this, but before you start writing one, we should settle on how the API will look. I'll review the existing discussion in detail over

[issue20193] Derby: Convert the zlib, _bz2 and _lzma modules to use Argument Clinic

2014-01-22 Thread Nadeem Vawda
Nadeem Vawda added the comment: The bz2 patch looks good to me, aside from a nit with the docstring for BZ2Compressor.__init__. The lzma patch produces a bunch of test failures for me. It looks like the __init__ methods for LZMACompressor and LZMADecompressor aren't accepting keyword args

[issue20193] Derby: Convert the zlib, _bz2 and _lzma modules to use Argument Clinic

2014-01-19 Thread Nadeem Vawda
Nadeem Vawda added the comment: The patches for bz2 and lzma look good to me, aside from one nit for lzma. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20193

[issue20182] Derby #13: Convert 50 sites to Argument Clinic across 5 files

2014-01-08 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20182 ___ ___ Python-bugs

[issue20184] Derby #16: Convert 50 sites to Argument Clinic across 9 files

2014-01-08 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20184 ___ ___ Python-bugs

[issue20185] Derby #17: Convert 50 sites to Argument Clinic across 14 files

2014-01-08 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20185 ___ ___ Python-bugs

[issue19885] lzma segfault when __init__ with non-existent file after executing the constructor (Python 2.7)

2013-12-04 Thread Nadeem Vawda
Nadeem Vawda added the comment: To clarify, which version(s) does this affect? I have not been able to reproduce against 3.4, and 2.7 does not included the lzma module in the first place. -- ___ Python tracker rep...@bugs.python.org http

[issue19878] bz2.BZ2File.__init__() cannot be called twice

2013-12-03 Thread Nadeem Vawda
Nadeem Vawda added the comment: It appears that this *does* affect 2.7 (though not 3.2, 3.3 or 3.4, fortunately): ~/src/cpython/2.7☿ gdb --ex run --args ./python -c 'import bz2; obj = bz2.BZ2File(/dev/null); obj.__init__()' «... snip banner ...» Starting program: /home.u/nadeem

[issue19839] bz2: regression wrt supporting files with trailing garbage after EOF

2013-12-01 Thread Nadeem Vawda
Nadeem Vawda added the comment: I'll have a patch for this in the next couple of days (and a similar one for the lzma module, which has the same issue (even though it's not a regression in that case)). In the meanwhile, you can work around this by feeding the compressed data

[issue19395] unpickled LZMACompressor is crashy

2013-10-28 Thread Nadeem Vawda
Nadeem Vawda added the comment: The part of this issue specific to LZMACompressor should now be fixed; I've filed issue 19425 for the issue with Pool.map hanging. -- resolution: - fixed stage: needs patch - committed/rejected status: open - closed

[issue19425] multiprocessing.Pool.map hangs if pickling argument raises an exception

2013-10-28 Thread Nadeem Vawda
New submission from Nadeem Vawda: [Split off from issue 19395] The following code hangs after hitting a TypeError trying to pickle one of the TextIOWrapper objects: import multiprocessing def read(f): return f.read() files = [open(path) for path in 3 * ['/dev/null']] pool

[issue19227] test_multiprocessing_xxx hangs under Gentoo buildbots

2013-10-28 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19227 ___ ___ Python-bugs

[issue19395] unpickled LZMACompressor is crashy

2013-10-26 Thread Nadeem Vawda
Nadeem Vawda added the comment: It looks like there's also a separate problem in the multiprocessing module. The following code hangs after hitting a TypeError trying to pickle one of the TextIOWrapper objects: import multiprocessing def read(f): return f.read() files = [open(path

[issue19395] unpickled LZMACompressor is crashy

2013-10-25 Thread Nadeem Vawda
Nadeem Vawda added the comment: As far as I can tell, liblzma provides no way to serialize a compressor's state, so the best we can do is raise a TypeError when attempting to pickle the LZMACompressor (and likewise for LZMADecompressor). Also, it's worth pointing out that the provided code

[issue19395] unpickled LZMACompressor is crashy

2013-10-25 Thread Nadeem Vawda
Nadeem Vawda added the comment: Yes, that's because the builtin map function doesn't handle each input in a separate process, so it uses the same LZMACompressor object everywhere. Whereas multiprocessing.Pool.map creates a new copy of the compressor object for each input, which is where

[issue19222] Add 'x' mode to gzip.open()

2013-10-18 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- assignee: - nadeem.vawda nosy: +nadeem.vawda resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue19223] Add 'x' mode to bz2.open()

2013-10-18 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- assignee: - nadeem.vawda resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19223

[issue19201] Add 'x' mode to lzma.open()

2013-10-18 Thread Nadeem Vawda
Nadeem Vawda added the comment: Fix committed. Thanks for the patches! As Jesús and Terry have said, this won't be backported to 3.3/2.7, since it is a new feature. [oylenshpeegul] | It's weird how different these three patches are! We're | essentially doing the same thing: please allow the x

[issue19201] Add 'x' mode to lzma.open()

2013-10-18 Thread Nadeem Vawda
Nadeem Vawda added the comment: [terry.reedy] | Arfrever's point about the order of characters makes me wonder why mode | strings (as opposed to characters in the strings) are being checked. | The following tests that exactly one of w, a, x appear in mode. | if len({'w', 'a', 'x'} set(mode

[issue18003] New lzma crazy slow with line-oriented reading.

2013-05-20 Thread Nadeem Vawda
Nadeem Vawda added the comment: No, that is the intended behavior for binary streams - they operate at the level of individual byes. If you want to treat your input file as Unicode-encoded text, you should open it in text mode. This will return a TextIOWrapper which handles the decoding and line

[issue18003] New lzma crazy slow with line-oriented reading.

2013-05-19 Thread Nadeem Vawda
Nadeem Vawda added the comment: I agree that making lzma.open() wrap its return value in a BufferedReader (or BufferedWriter, as appropriate) is the way to go. I'm currently travelling and don't have my SSH key with me - Serhiy, can you make the change? I'll put together a documentation patch

[issue18003] New lzma crazy slow with line-oriented reading.

2013-05-19 Thread Nadeem Vawda
Nadeem Vawda added the comment: I agree that making lzma.open() wrap its return value in a BufferedReader (or BufferedWriter, as appropriate) is the way to go. On second thoughts, there's no need to change the behavior for mode='wb'. We can just return a BufferedReader for mode='rb

[issue18003] New lzma crazy slow with line-oriented reading.

2013-05-18 Thread Nadeem Vawda
Nadeem Vawda added the comment: Have you tried running the benchmark against the default (3.4) branch? There was some significant optimization work done in issue 16034, but the changes were not backported to 3.3. -- ___ Python tracker rep

[issue17843] Lib/test/testbz2_bigmem.bz2 trigger virus warnings

2013-04-30 Thread Nadeem Vawda
Nadeem Vawda added the comment: Benjamin, please cherry-pick this for 2.7.4 as well (changesets b7bfedc8ee18 and 529c4defbfd7). -- stage: needs patch - commit review versions: +Python 2.7 ___ Python tracker rep...@bugs.python.org http

[issue17843] Lib/test/testbz2_bigmem.bz2 trigger virus warnings

2013-04-28 Thread Nadeem Vawda
Nadeem Vawda added the comment: OK, 2.7 is done. Georg, what do we want to do for 3.2? I've attached a patch. -- assignee: nadeem.vawda - georg.brandl keywords: +patch Added file: http://bugs.python.org/file30049/bz2-viruswarning.diff ___ Python

[issue17843] Lib/test/testbz2_bigmem.bz2 trigger virus warnings

2013-04-25 Thread Nadeem Vawda
Nadeem Vawda added the comment: Oh dear. I'll update the test suite over the weekend. In the meanwhile, Christian, can you confirm which versions are affected? The file should only have been included in 2.7 and 3.2. -- assignee: - nadeem.vawda

[issue14398] bz2.BZ2DEcompressor.decompress fail on large files

2013-04-21 Thread Nadeem Vawda
Nadeem Vawda added the comment: Hmm, so actually most of the bugs fixed in 2.7 and 3.2 weren't present in 3.3 and 3.4, and those versions already had tests equivalent to the tests I added for 2.7/3.2. As for the changes that I did make to 3.3/3.4: - two of the three cover cases that only occur

[issue14398] bz2.BZ2DEcompressor.decompress fail on large files

2013-04-18 Thread Nadeem Vawda
Nadeem Vawda added the comment: An oversight on my part, I think. I'll add tests for 3.x this weekend. -- status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14398

[issue13898] Ignored exception in test_ssl

2013-03-03 Thread Nadeem Vawda
Nadeem Vawda added the comment: This change fixes the problem (and doesn't break anything else that I can see): --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -979,7 +979,7 @@ self.sslconn = self.server.context.wrap_socket( self.sock

[issue13898] Ignored exception in test_ssl

2013-03-03 Thread Nadeem Vawda
Nadeem Vawda added the comment: You could add a comment explaining the issue. Done. This doesn't seem to affect 2.7. Marking as fixed in 3.2/3.3/3.4. -- resolution: - fixed stage: patch review - committed/rejected status: open - closed versions: -Python 2.7

[issue13886] readline-related test_builtin failure

2013-02-02 Thread Nadeem Vawda
Nadeem Vawda added the comment: You're right; it breaks backspacing over multibyte characters. I should have tested it more carefully before committing. I'll revert the changes. -- resolution: fixed - stage: committed/rejected - needs patch status: closed - open

[issue1159051] Handle corrupted gzip files with unexpected EOF

2013-02-02 Thread Nadeem Vawda
Nadeem Vawda added the comment: I think the new behavior should be controlled by a constructor flag, maybe named defer_errors. I don't like the idea of adding the flag to read(), since that makes us diverge from the standard file interface. Making a distinction between size0 and size=None seems

[issue13886] readline-related test_builtin failure

2013-01-27 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- assignee: - nadeem.vawda resolution: - fixed stage: patch review - committed/rejected status: open - closed versions: +Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue1159051] Handle corrupted gzip files with unexpected EOF

2013-01-20 Thread Nadeem Vawda
Nadeem Vawda added the comment: The updated patch looks good to me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1159051 ___ ___ Python-bugs

[issue1159051] Handle corrupted gzip files with unexpected EOF

2013-01-19 Thread Nadeem Vawda
Nadeem Vawda added the comment: I've reviewed the patch and posted some comments on Rietveld. I doubt about backward compatibility. It's obvious that struct.error and TypeError are unintentional, and EOFError is purposed for this case. However users can catch undocumented but de facto

[issue15955] gzip, bz2, lzma: add option to limit output size

2013-01-19 Thread Nadeem Vawda
Nadeem Vawda added the comment: What if unconsumed_tail is not empty but less than needed to decompress at least one byte? We need read more data until unconsumed_tail grow enought to be decompressed. This is possible in zlib, but not in bz2. According to the manual [1], it is perfectly OK

[issue16943] seriously? FileCookieJar can't really save ? save method is NotImplemented

2013-01-12 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- resolution: - duplicate stage: - committed/rejected status: open - closed superseder: - seriously? urllib still doesn't support persistent connections? ___ Python tracker rep...@bugs.python.org

[issue16828] bz2 error on compression of empty string

2013-01-02 Thread Nadeem Vawda
Nadeem Vawda added the comment: Fixed. Thanks for the bug report and the patches! -- assignee: - nadeem.vawda keywords: +3.3regression -patch resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep

[issue15955] gzip, bz2, lzma: add option to limit output size

2012-12-09 Thread Nadeem Vawda
Nadeem Vawda added the comment: # Using zlib's interface while not d.eof: compressed = d.unconsumed_tail or f.read(8192) if not compressed: raise ValueError('End-of-stream marker not found') output = d.decompress(compressed, 8192

[issue15955] gzip, bz2, lzma: add option to limit output size

2012-12-02 Thread Nadeem Vawda
Nadeem Vawda added the comment: I've tried reimplementing LZMAFile in terms of the decompress_into() method, and it has ended up not being any faster than the existing implementation. (It is _slightly_ faster for readinto() with a large buffer size, but all other cases it was either of equal

[issue15677] Gzip/zlib allows for compression level=0

2012-11-11 Thread Nadeem Vawda
Nadeem Vawda added the comment: Committed. Thanks for the patch! -- resolution: - fixed stage: commit review - committed/rejected status: open - closed type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15677

[issue16350] zlib.Decompress.decompress() after EOF discards existing value of unused_data

2012-11-10 Thread Nadeem Vawda
Nadeem Vawda added the comment: New patch committed. Once again, thanks for all your work on this issue! -- stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16350

[issue16411] zlib.Decompress.decompress() retains pointer to input buffer without acquiring reference to it

2012-11-10 Thread Nadeem Vawda
Nadeem Vawda added the comment: Ah, that's much nicer than either of my ideas. Patch committed. Thanks! -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http

[issue16441] range usage in gzip module leads to excessive memory usage.

2012-11-08 Thread Nadeem Vawda
Nadeem Vawda added the comment: Looks good to me. Go ahead. You needn't add or change any tests for this, but you should run the existing tests before committing, just to be safe. -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http

[issue15955] gzip, bz2, lzma: add option to limit output size

2012-11-06 Thread Nadeem Vawda
Nadeem Vawda added the comment: I suspect that it will be slower than the decompress_into() approach, but as you say, we need to do benchmarks to see for sure. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15955

[issue16350] zlib.Decompress.decompress() after EOF discards existing value of unused_data

2012-11-06 Thread Nadeem Vawda
Nadeem Vawda added the comment: These were not idle questions. I wrote the patch, and I had to know what behavior is correct. Ah, sorry. I assumed you were going to submit a separate patch to fix the unconsumed_tail issues. Here's the patch. It fixes potential memory bug (unconsumed_tail

[issue16350] zlib.Decompress.decompress() after EOF discards existing value of unused_data

2012-11-04 Thread Nadeem Vawda
Nadeem Vawda added the comment: Fixed. Thanks for the patch! This hacking is not needed, if first argument of PyBytes_FromStringAndSize() is NULL, the contents of the bytes object are uninitialized. Oh, cool. I didn't know about that. What should unconsumed_tail be equal after EOF? b

[issue16411] zlib.Decompress.decompress() retains pointer to input buffer without acquiring reference to it

2012-11-04 Thread Nadeem Vawda
New submission from Nadeem Vawda: When calling zlib.Decompress.decompress() with a max_length argument, if the input data is not full consumed, the next_in pointer in the z_stream struct are left pointing into the data object, but the decompressor does not hold a reference to this object

[issue16350] zlib.Decompress.decompress() after EOF discards existing value of unused_data

2012-11-04 Thread Nadeem Vawda
Nadeem Vawda added the comment: flush() does not update unconsumed_tail and unused_data. import zlib x = zlib.compress(b'abcdefghijklmnopqrstuvwxyz') + b'0123456789' dco = zlib.decompressobj() dco.decompress(x, 1) b'a' dco.flush() b'bcdefghijklmnopqrstuvwxyz' dco.unconsumed_tail

[issue15955] gzip, bz2, lzma: add option to limit output size

2012-11-04 Thread Nadeem Vawda
Nadeem Vawda added the comment: I agree that being able to limit output size is useful and desirable, but I'm not keen on copying the max_length/unconsumed_tail approach used by zlib's decompressor class. It feels awkward to use, and it complicates the implementation of the existing decompress

[issue16316] Support xz compression in mimetypes module

2012-10-28 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16316

[issue16350] zlib.Decompress.decompress() after EOF discards existing value of unused_data

2012-10-28 Thread Nadeem Vawda
New submission from Nadeem Vawda: From issue 5210: amaury.forgeotdarc wrote: Hm, I tried a modified version of your first test, and I found another problem with the current zlib library; starting with the input: x = x1 + x2 + HAMLET_SCENE# both compressed and uncompressed data

[issue5210] zlib does not indicate end of compressed stream properly

2012-10-28 Thread Nadeem Vawda
Nadeem Vawda added the comment: This bug (zlib not providing a way to detect end-of-stream) has already been fixed - see issue 12646. I've opened issue 16350 for the unused_data problem. -- resolution: - out of date stage: test needed - committed/rejected status: open - closed

[issue16350] zlib.Decompress.decompress() after EOF discards existing value of unused_data

2012-10-28 Thread Nadeem Vawda
Nadeem Vawda added the comment: Interesting idea, but I'm not sure it would be worth the effort. It would make the code and API more complicated, so it wouldn't really help users, and would be an added maintenance burden. -- ___ Python tracker rep

[issue12692] test_urllib2net is triggering a ResourceWarning

2012-10-21 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12692

[issue5148] gzip.open breaks with 'U' flag

2012-10-21 Thread Nadeem Vawda
Nadeem Vawda added the comment: The data corruption issue is now fixed in the 2.7 branch. In 3.x, using a mode containing 'U' results in an exception rather than silent data corruption. Additionally, gzip.open() has supported text modes (rt/wt/at) and newline translation since 3.3 [issue

[issue14398] bz2.BZ2DEcompressor.decompress fail on large files

2012-10-21 Thread Nadeem Vawda
Nadeem Vawda added the comment: I'm working on it now. Will push in the next 15 minutes or so. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14398

[issue14398] bz2.BZ2DEcompressor.decompress fail on large files

2012-10-21 Thread Nadeem Vawda
Nadeem Vawda added the comment: All fixed, along with some other similar but harder-to-trigger bugs. Thanks for the bug report, Laurent! -- resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep

[issue10050] urllib.request still has old 2.x urllib primitives

2012-10-14 Thread Nadeem Vawda
Nadeem Vawda added the comment: Hmm, OK. URLopener and FancyURLopener do each issue a DeprecationWarning when used, though. If they are not actually deprecated, perhaps we should remove the warnings for the moment? -- ___ Python tracker rep

[issue14214] test_concurrent_futures hangs

2012-10-13 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- resolution: - works for me stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14214

[issue14120] ARM Ubuntu 3.x buildbot failing test_dbm

2012-10-13 Thread Nadeem Vawda
Nadeem Vawda added the comment: No sign of these failures any more; looks like that fixed it. -- resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue14229] On KeyboardInterrupt, the exit code should mirror the signal number

2012-10-13 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- resolution: - rejected stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14229

[issue10050] urllib.request still has old 2.x urllib primitives

2012-10-13 Thread Nadeem Vawda
Nadeem Vawda added the comment: Are we still planning on removing URLopener and FancyURLopener in 3.4? The documentation for 3.3 does not list these classes as deprecated. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue16034] bz2 module appears slower in Python 3.x versus Python 2.x

2012-10-08 Thread Nadeem Vawda
Nadeem Vawda added the comment: I've released v0.95 of bz2file, which incorporates all the optimizations discussed here. The performance should be similar to 2.x's bz2 in most cases. It is still a lot slower when calling read(10) or read(1), but I hope no-one is doing that anywhere where

[issue16034] bz2 module appears slower in Python 3.x versus Python 2.x

2012-10-01 Thread Nadeem Vawda
Nadeem Vawda added the comment: Ah, nice - I didn't think of that optimization. Neater and faster. I've committed this patch [e6d872b61c57], along with a minor bugfix [7252f9f95fe6], and another optimization for readline()/readlines() [6d7bf512e0c3]. [merge with default: a19f47d380d2

[issue16034] bz2 module appears slower in Python 3.x versus Python 2.x

2012-09-30 Thread Nadeem Vawda
Nadeem Vawda added the comment: Yes, of course. Awesome. I plan to do a new release for this in the next couple of days. We can even speed up 1.5 times the reading of small chunks, if we inline _check_can_read() and _read_block(). Interesting idea, but I don't think it's worthwhile

[issue16034] bz2 module appears slower in Python 3.x versus Python 2.x

2012-09-30 Thread Nadeem Vawda
Nadeem Vawda added the comment: Recursive inline _check_can_read() will be enough. Now this check calls 4 Python functions (_check_can_read(), readable(), _check_non_closed(), closed). Recursive inlining only readable() in _check_can_read() is achieved significant but less (about 30

[issue16034] bz2 module appears slower in Python 3.x versus Python 2.x

2012-09-29 Thread Nadeem Vawda
Nadeem Vawda added the comment: Thanks for the bug report, Victor, and thank you Serhiy for the patch! Serhiy, would you be OK with me also including this patch in the bz2file package? -- resolution: - fixed stage: - committed/rejected status: open - closed versions: +Python 3.4

[issue15955] gzip, bz2, lzma: add method to get decompressed size

2012-09-23 Thread Nadeem Vawda
Nadeem Vawda added the comment: As far as I can tell, there is no way to find this out reliably without decompressing the entire file. With gzip, the file trailer contains the uncompressed size modulo 2^32, but this seems less than useful. It appears that the other two formats do not store

[issue15666] PEP 3121, 384 refactoring applied to lzma module

2012-08-15 Thread Nadeem Vawda
Nadeem Vawda added the comment: Thanks for the patch. Unfortunately I don't have much free time at the moment, so it might be a few weeks before I get a chance to review it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue15664] test_curses not run with 'make test'

2012-08-15 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- superseder: - test_curses skipped on buildbots ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15664

[issue12669] test_curses skipped on buildbots

2012-08-15 Thread Nadeem Vawda
Nadeem Vawda added the comment: Nadeem: is the failure you show in msg141798 with a version of test_curses that uses pty.openpty? Yes, I tried the following change: --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -328,11 +328,12 @@ curses.resetty

[issue12669] test_curses skipped on buildbots

2012-08-15 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- stage: - needs patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12669 ___ ___ Python-bugs

[issue15654] PEP 384 Refactoring applied to bz2 module

2012-08-14 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15654 ___ ___ Python-bugs

[issue15546] Iteration breaks with bz2.open(filename,'rt')

2012-08-05 Thread Nadeem Vawda
Nadeem Vawda added the comment: Before these fixes, it looks like all three classes' peek() methods were susceptible to the same problem as read1(). The fixes for BZ2File.read1() and LZMAFile.read1() should have fixed peek() as well; both methods are implemented in terms of _fill_buffer

[issue15546] Iteration breaks with bz2.open(filename,'rt')

2012-08-05 Thread Nadeem Vawda
Nadeem Vawda added the comment: No, if _read() is called once the file is already at EOF, it raises an EOFError (http://hg.python.org/cpython/file/8c07ff7f882f/Lib/gzip.py#l433), which will then break out of the loop. -- ___ Python tracker rep

[issue15546] Iteration breaks with bz2.open(filename,'rt')

2012-08-04 Thread Nadeem Vawda
Nadeem Vawda added the comment: OK, BZ2File should now be fixed. It looks like LZMAFile and GzipFile may be susceptible to the same problem; I'll push fixes for them shortly. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue15546] Iteration breaks with bz2.open(filename,'rt')

2012-08-04 Thread Nadeem Vawda
Nadeem Vawda added the comment: Done. Thanks for the bug report, David. -- resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15546

[issue15546] Iteration breaks with bz2.open(filename,'rt')

2012-08-03 Thread Nadeem Vawda
Nadeem Vawda added the comment: I can't seem to reproduce this with an up-to-date checkout from Mercurial: import bz2 g = bz2.open('access-log-0108.bz2','rt') next(g) '140.180.132.213 - - [24/Feb/2008:00:08:59 -0600] GET /ply/ply.html HTTP/1.1 200 97238\n' (where 'access

[issue15546] Iteration breaks with bz2.open(filename,'rt')

2012-08-03 Thread Nadeem Vawda
Nadeem Vawda added the comment: The cause of this problem is that BZ2File.read1() sometimes returns b, even though the file is not at EOF. This happens when the underlying BZ2Decompressor cannot produce any decompressed data from just the block passed to it in _fill_buffer(); in this case

[issue15405] Invitation to connect on LinkedIn

2012-07-20 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- resolution: - invalid stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15405

[issue15204] Deprecate the 'U' open mode

2012-06-28 Thread Nadeem Vawda
Nadeem Vawda nadeem.va...@gmail.com added the comment: +1 for the general idea of deprecating and eventually removing the U modes. But I agree with David, that it doesn't make sense to have separate steps for 3.5 and 3.6/4.0. If you make the code raise an exception when U is used, how

[issue13876] Sporadic failure in test_socket

2012-06-27 Thread Nadeem Vawda
Nadeem Vawda nadeem.va...@gmail.com added the comment: Merging nosy list from duplicate issue 15155. -- nosy: +giampaolo.rodola, neologix, pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13876

[issue12559] gzip.open() needs an optional encoding argument

2012-06-26 Thread Nadeem Vawda
Nadeem Vawda nadeem.va...@gmail.com added the comment: I already fixed this without knowing about this issue; see 55202ca694d7. storchaka: Why not use io.TextWrapper? I think it is the right answer for this issue. The proposed patch (and the code I committed) *do* use TextIOWrapper. Unless

[issue10376] ZipFile unzip is unbuffered

2012-06-23 Thread Nadeem Vawda
Nadeem Vawda nadeem.va...@gmail.com added the comment: Patch looks fine to me. Antoine, can you commit this? I'm currently away from the computer that has my SSH key on it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue14684] zlib set dictionary support inflateSetDictionary

2012-06-21 Thread Nadeem Vawda
Nadeem Vawda nadeem.va...@gmail.com added the comment: Just saw this on the checkins list; where are the other options documented? They aren't, AFAIK. I've been planning on adding them when I've got time (based on the zlib manual at http://zlib.net/manual.html), but with the upcoming feature

[issue14684] zlib set dictionary support inflateSetDictionary

2012-06-20 Thread Nadeem Vawda
Nadeem Vawda nadeem.va...@gmail.com added the comment: Committed. Once again, thanks for the patch! -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http

[issue14684] zlib set dictionary support inflateSetDictionary

2012-06-19 Thread Nadeem Vawda
Nadeem Vawda nadeem.va...@gmail.com added the comment: To restate my position: the need is for an immutable string of bytes, [...] I disagree that we should require the dictionary to be immutable - if the caller wishes to use a mutable buffer here, it is their responsibility to ensure

[issue14684] zlib set dictionary support inflateSetDictionary

2012-06-19 Thread Nadeem Vawda
Nadeem Vawda nadeem.va...@gmail.com added the comment: I plan to commit it (along with the buffer API changes) tomorrow. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14684

Re: python 3.3 bz2 decompression testing results

2012-06-18 Thread Nadeem Vawda
Hi Pauli, Thank you for your interest in improving the bz2 module. However, I'm not sure of what you are saying in your email. If you believe you have found a bug in the module, then please provide clear instructions on how to reproduce the error(s), preferably using just one data file that

  1   2   3   4   5   6   >