[issue4263] BufferedWriter non-blocking overage
Sever Băneșiu banesiu.se...@gmail.com added the comment: Looks like the test covering the pre-flush condition is missing. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Benjamin Peterson benja...@python.org added the comment: 2009/3/5 Sever Băneșiu rep...@bugs.python.org: Sever Băneșiu banesiu.se...@gmail.com added the comment: Looks like the test covering the pre-flush condition is missing. That test is no longer applicable because max_buffer_size is deprecated and unused. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Benjamin Peterson benja...@python.org added the comment: This has been fixed in io-c branch. (r70152) -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Benjamin Peterson benja...@python.org added the comment: This has been cured in the io-c branch. -- nosy: +benjamin.peterson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Sever Băneșiu banesiu.se...@gmail.com added the comment: Anyway :) Practically, the test does work on both py3k and another implementation, so I don't see any urgency to remove anything from it. Indeed, it doesn't hurt keeping it. For completeness' sake I've updated your tests to cover the pre-flush condition existent in the python implementation. Theoretically this should be the desired behavior in any other implementation. Added file: http://bugs.python.org/file12636/nonblocking2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Sever Băneșiu banesiu.se...@gmail.com added the comment: Thanks for the new implementation of MockNonBlockWriterIO class. It makes tests so much easier to read. There are some minor things in your patch that I would change. For example: # 1 byte will be written, the rest will be buffered raw.block_on(bk) self.assertEquals(bufio.write(bjklmn), 5) # ... raw.block_on(b0) The comment is misleading because in fact no byte is written at raw level. That's because the data size is smaller than the buffer size and the buffer is empty (was emptied by the last write call). All this renders raw.block_on(bk) call useless. I also think this is the correct behavior regardless of implementation language of BufferedWriter class i.e. no write call should write at raw level smaller chunks of data than buffer's size unless it has to. Your tests can't cover the pre-flush condition because max_buffer_size equals buffer_size. Unless you'll beat me to it or prove me wrong, I'll update your patch next year. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Antoine Pitrou pit...@free.fr added the comment: The comment is misleading because in fact no byte is written at raw level. That's because the data size is smaller than the buffer size and the buffer is empty (was emptied by the last write call). It depends on the implementation. A different implementation may use a different algorithm. I also think this is the correct behavior regardless of implementation language of BufferedWriter class i.e. no write call should write at raw level smaller chunks of data than buffer's size unless it has to. But how do you decide when it has to? Unless you want to constrain the exact implemented algorithm, you can't do that in your tests. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Sever Băneșiu banesiu.se...@gmail.com added the comment: The comment is misleading because in fact no byte is written at raw level. That's because the data size is smaller than the buffer size and the buffer is empty (was emptied by the last write call). It depends on the implementation. A different implementation may use a different algorithm. I feel that no matter what implementation algorithm BufferedWriter uses it shouldn't write smaller chunks of data than buffer's size or else the buffer is useless. I also think this is the correct behavior regardless of implementation language of BufferedWriter class i.e. no write call should write at raw level smaller chunks of data than buffer's size unless it has to. But how do you decide when it has to? Unless you want to constrain the exact implemented algorithm, you can't do that in your tests. When a direct or indirect (e.g. on close) flush is called for the file object. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Antoine Pitrou pit...@free.fr added the comment: I feel that no matter what implementation algorithm BufferedWriter uses it shouldn't write smaller chunks of data than buffer's size or else the buffer is useless. If you rewrite the above sentence using the word statistically, then I can agree :) But if I look at e.g. the fwrite() manpage, I see no guarantee that the stdio layer will never make a call to write() with a size smaller than the buffer size. The buffered layer should be free to manage its buffer in what it believes is the most efficient way. The only guarantee is that it won't buffer more than max_buffer_size. Anyway :) Practically, the test does work on both py3k and another implementation, so I don't see any urgency to remove anything from it. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Antoine Pitrou pit...@free.fr added the comment: The tests should be written so as not to rely on internal implementation details (the _write_buf attribute). -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Antoine Pitrou pit...@free.fr added the comment: Here is a patch which replaces testWriteNonBlocking with a reasonable implementation-independent test (it works with e.g. the io-c sandbox). The new test also checks for the current problem, i.e. it passes with the fix to io.py and fails without. Added file: http://bugs.python.org/file12498/nonblocking.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Sever Băneșiu [EMAIL PROTECTED] added the comment: Christian, if the patch looks good to you I think now it's a good time to commit it. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: The patch is good. I was first surprised by the fact that e.characters_written is not used in the write() method; but _flush_unlocked() already adjusts the _write_buf according to the original e.characters_written raised by the underlying raw file. Everything is fine. I suggest however to add some tests around the first except BlockingIOError. This would answer the question: # XXX Why not just let the exception pass through? For example, I modified a function in your patch: def testWriteNonBlockingOverage(self): raw = MockNonBlockWriterIO((-1, -2)) [...] # Subsequent calls to write() try to flush the raw file. try: bufio.write(bx) except io.BlockingIOError as e: # Two more chars were written at the raw level self.assertEqual(bufio._write_buf, write_buf[2:]) # But write() did not accept anything. self.assertEqual(e.characters_written, 0) else: self.fail(BlockingIOError not raised) -- nosy: +amaury.forgeotdarc ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Banesiu Sever [EMAIL PROTECTED] added the comment: Thanks for your review, here's a new patch. I've added a new test for the pre-flush condition and made the comments less cryptic. Added file: http://bugs.python.org/file11953/bw_overage2.diff ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Christian Heimes [EMAIL PROTECTED] added the comment: We have discussed this bug in the python developer chat yesterday. I decided to wait until after the 3.0.0 release. The problem is not critical enough for 3.0.0. I like to keep the amount of changes during the RC phase to a minimum. -- nosy: +christian.heimes priority: - normal stage: - patch review type: - behavior versions: -Python 3.1 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: I do concur with the desire to restrict changes during RC phase. Do this also mean that merges from trunk will be reduced to the strict minimum? No global merge, only on a revision basis after review. In this case we could apply the patch to the trunk, and let a future global merge propagate the change to py3k. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4263] BufferedWriter non-blocking overage
New submission from Banesiu Sever [EMAIL PROTECTED]: In some corner cases io.BufferedWriter raises an io.BlockingIOError lying about the number of characters written. I've added some tests and a small change to fix this issue. -- components: Library (Lib) files: bw_overage.diff keywords: patch messages: 75523 nosy: banesiu.sever severity: normal status: open title: BufferedWriter non-blocking overage versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1 Added file: http://bugs.python.org/file11945/bw_overage.diff ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue4263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
