[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: r67380 has the fix. Thanks for the review, Skip! -- resolution: -> fixed status: open -> closed ___ Python tracker <[EMAIL PROTECTED]>

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: On Mon, Nov 24, 2008 at 16:25, Skip Montanaro <[EMAIL PROTECTED]> wrote: > > Skip Montanaro <[EMAIL PROTECTED]> added the comment: > > Brett> I still need a review for doc_dbm_strings.diff, though, which >Brett> clarifies the docs, fixes one

[issue3799] Byte/string inconsistencies between different dbm modules

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Brett> I still need a review for doc_dbm_strings.diff, though, which Brett> clarifies the docs, fixes one oversight in dbm.dumb, and extends Brett> testing to make sure strings can be accepted. Was my comment http://bugs.python.o

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Brett Cannon <[EMAIL PROTECTED]>: -- components: +Library (Lib) stage: needs patch -> commit review ___ Python tracker <[EMAIL PROTECTED]> ___ __

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: specify_open_encoding.diff has been committed in r67369. I still need a review for doc_dbm_strings.diff, though, which clarifies the docs, fixes one oversight in dbm.dumb, and extends testing to make sure strings can be accepted. _

[issue3799] Byte/string inconsistencies between different dbm modules

Skip Montanaro <[EMAIL PROTECTED]> added the comment: py3k patched with specify_open_encoding.diff passes test_dbm_dumb on my Mac (Leopard, Intel). Might as well assign this to Brett. He seems to be doing all the heavy lifting anyway. ;-) Skip -- assignee: -> brett.cannon __

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: On Fri, Nov 21, 2008 at 11:01, Skip Montanaro <[EMAIL PROTECTED]> wrote: > > Skip Montanaro <[EMAIL PROTECTED]> added the comment: > > One doc nit: There is still reference to ``gdbm`` and Dbm (or dbm) objects > when they should probably use ``d

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by STINNER Victor <[EMAIL PROTECTED]>: -- nosy: -haypo ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list mailing

[issue3799] Byte/string inconsistencies between different dbm modules

Skip Montanaro <[EMAIL PROTECTED]> added the comment: One doc nit: There is still reference to ``gdbm`` and Dbm (or dbm) objects when they should probably use ``dbm.gnu`` and ``dbm.ndbm``, respectively. I'm confused by the encoding="Latin-1" args to _io.open for dbm.dumb. I thought the default

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: On Fri, Nov 21, 2008 at 10:32, Skip Montanaro <[EMAIL PROTECTED]> wrote: > > Skip Montanaro <[EMAIL PROTECTED]> added the comment: > > Brett> In other words I think my solution works and pickle is the >Brett> trouble-maker in all of this. >

[issue3799] Byte/string inconsistencies between different dbm modules

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Brett> In other words I think my solution works and pickle is the Brett> trouble-maker in all of this. I can buy that. Should pickle map "copy_reg" to "copyreg"? Is that the only incompatibility? Actually, it seems this ticket should b

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Brett Cannon <[EMAIL PROTECTED]>: -- resolution: accepted -> ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Brett Cannon <[EMAIL PROTECTED]>: Added file: http://bugs.python.org/file12100/mydb3read.py ___ Python tracker <[EMAIL PROTECTED]> ___ __

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Brett Cannon <[EMAIL PROTECTED]>: Added file: http://bugs.python.org/file12099/mydb2write.py ___ Python tracker <[EMAIL PROTECTED]> ___ _

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Brett Cannon <[EMAIL PROTECTED]>: Added file: http://bugs.python.org/file12098/pickle3read.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: So the use of pickle is not fair as that doesn't round-trip if you simply write out a file because py3k pickle doesn't like classic classes and the new-style classes want copy_reg which is not in existence in py3k since it was renamed. See pickl

[issue3799] Byte/string inconsistencies between different dbm modules

Guido van Rossum <[EMAIL PROTECTED]> added the comment: [fix title] -- title: Re: r67310 - in python/branches/py3k: Lib/dbm/dumb.py Lib/test/test_dbm_dumb.py Misc/NEWS -> Byte/string inconsistencies between different dbm modules ___ Python tracker <

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Skip Montanaro <[EMAIL PROTECTED]>: Added file: http://bugs.python.org/file12096/mydb3read.py ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3799] Byte/string inconsistencies between different dbm modules

Skip Montanaro <[EMAIL PROTECTED]> added the comment: damn... my cc to [EMAIL PROTECTED] didn't work. Here's the recap (message to python-checkins): me> ... I thought Guido was of the opinion that the 3.0 version should me> be able to read dumb dbms written by earlier Python versions..

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Barry A. Warsaw <[EMAIL PROTECTED]>: -- priority: deferred blocker -> release blocker ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: Have another patch that fixes all open() calls to specify the file encoding in dbm.dumb. Also caught one spot in _addkey() where decode("Latin-1") was not being called. Added file: http://bugs.python.org/file12085/specify_open_encoding.diff __

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: I am re-opening this as a deferred blocker with a patch to document that dbm implementations write out and return bytes, but that strings are accepted and implicitly converted. -- priority: release blocker -> deferred blocker status: cl

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: r67310 has the fix. -- resolution: -> accepted status: open -> closed ___ Python tracker <[EMAIL PROTECTED]> ___ __

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Brett Cannon <[EMAIL PROTECTED]>: -- keywords: +needs review stage: -> commit review ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: I have attached a file that does everything internally as UTF-8 but reads and writes to disk as Latin-1. I added some unit tests to verify that taking a character encoded in UTF-8 or as a string still works properly regardless of whether one use

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: OK, now I see why it is called 'dumb'; the thing literally just dumps out the repr of two strings on each line for key/value pairs. To read it just evals each line in the string. And whichdb detects this format by looking for ' or " as the first

[issue3799] Byte/string inconsistencies between different dbm modules

Brett Cannon <[EMAIL PROTECTED]> added the comment: If you look at the 2.7 code all it requires of keys and values in __setitem__ is that they are strings; there is nothing about Latin-1 in terms of specific encoding (must be a 3.0 addition to make the str/unicode transition the easiest). That wo

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Barry A. Warsaw <[EMAIL PROTECTED]>: -- priority: deferred blocker -> release blocker ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Skip Montanaro <[EMAIL PROTECTED]>: -- nosy: -skip.montanaro ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list

[issue3799] Byte/string inconsistencies between different dbm modules

Barry A. Warsaw <[EMAIL PROTECTED]> added the comment: I don't see enough progress on this issue, and I'm not going to hold up 3.0rc2 for it, so I'm deferring. -- nosy: +barry priority: release blocker -> deferred blocker ___ Python tracker <[EMAIL PR

[issue3799] Byte/string inconsistencies between different dbm modules

Christian Heimes <[EMAIL PROTECTED]> added the comment: The patch causes three errors: == ERROR: test_anydbm_access (test.test_dbm.TestCase-dbm.dumb) -- Traceba

[issue3799] Byte/string inconsistencies between different dbm modules

STINNER Victor <[EMAIL PROTECTED]> added the comment: For information, Python3 trunk fails on: test.support.TestFailed: Traceback (most recent call last): File "Lib/test/test_dbm.py", line 157, in test_keys self.assert_('xxx' not in self.d) TypeError: gdbm key must be bytes, not str --

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Benoit Boissinot <[EMAIL PROTECTED]>: -- nosy: -bboissin ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list mai

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Benoit Boissinot <[EMAIL PROTECTED]>: ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Benoit Boissinot <[EMAIL PROTECTED]>: -- nosy: +bboissin ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list mail

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Jesús Cea Avión <[EMAIL PROTECTED]>: -- nosy: +jcea ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Python-bugs-list mailing l

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Barry A. Warsaw <[EMAIL PROTECTED]>: -- priority: deferred blocker -> release blocker ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Barry A. Warsaw <[EMAIL PROTECTED]>: -- priority: release blocker -> deferred blocker ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Barry A. Warsaw <[EMAIL PROTECTED]>: -- priority: deferred blocker -> release blocker ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3799] Byte/string inconsistencies between different dbm modules

Guido van Rossum <[EMAIL PROTECTED]> added the comment: I think this isn't quite right. Ideally a fix should maintain several important properties: (1) Be able to read databases written by Python 2.x. (1a) Write databases readable by Python 2.x. (2) Use the same mapping between str and bytes

[issue3799] Byte/string inconsistencies between different dbm modules

Changes by Barry A. Warsaw <[EMAIL PROTECTED]>: -- priority: release blocker -> deferred blocker ___ Python tracker <[EMAIL PROTECTED]> ___

[issue3799] Byte/string inconsistencies between different dbm modules

Skip Montanaro <[EMAIL PROTECTED]> added the comment: I'm not sure. I've never done anything with the io module. Simply eliminating the bytes checks and letting it try to write the string yields: File "/Users/skip/local/lib/python3.0/dbm/dumb.py", line 170, in __setitem__ self._addkey(k

[issue3799] Byte/string inconsistencies between different dbm modules

Guido van Rossum <[EMAIL PROTECTED]> added the comment: How hard would it be to fix dbm.dumb to accept strings as well? ___ Python tracker <[EMAIL PROTECTED]> ___ __

[issue3799] Byte/string inconsistencies between different dbm modules

Skip Montanaro <[EMAIL PROTECTED]> added the comment: Extra data point. I tried f["1"] = "a" and f[b"1"] = "a" with dbm.{gnu,ndbm,dumb,sqlite}. All worked with bytes. A except dbm.dumb worked with strings. (dbm.sqlite is my little proof-of-concept module currently in the sandbox.) __

[issue3799] Byte/string inconsistencies between different dbm modules

Guido van Rossum <[EMAIL PROTECTED]> added the comment: Making this into a release blocker just so someone will look at it. Needs to be decided & fixed by rc2 at the very latest. -- nosy: +gvanrossum priority: high -> release blocker ___ Python tracke

[issue3799] Byte/string inconsistencies between different dbm modules

New submission from Skip Montanaro <[EMAIL PROTECTED]>: Consider these two timeit commands: py3k% python3.0 -m timeit -s 'import dbm.ndbm as db' -s 'f = db.open("/tmp/trash.db", "c")' 'for i in range(1000): f[str(i)] = str(i)' 100 loops, best of 3: 5.51 msec per loop py3k% python3.0 -m timeit -