On 22/02/2011 12:14, Jesus Cea wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I have 10MB pickled structure generated in Python 2.7. I only use basic
types (no clases) like sets, dictionaries, lists, strings, etc.
The pickle stores a lot of strings. Some of them should be "bytes",
while other should be "unicode". My idea is to import ALL the strings as
bytes in Python 3.2 and navigate the data structure to convert the
appropiate values to unicode, in a one-time operation (I version the
structure, so I can know if this conversion is already done, simply
storing a new version value).
But I get this error:
"""
Python 3.2 (r32:88445, Feb 21 2011, 13:34:07)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
f=open("file.pickle", mode="rb").read()
len(f)
9847316
b=pickle.loads(f,encoding="latin1")
Traceback (most recent call last):
File "<stdin>", line 1, in<module>
ValueError: operation forbidden on released memoryview object
"""
That seems like an odd error, but the decision was made that Python 2
byte-strings would be unpickled on Python 3 as Unicode strings.
See the discussion here:
http://bugs.python.org/issue6784
This is basically because many people "do the wrong thing" and use
Python 2 byte strings for restoring text. What it means though is that
people who "do the right thing" and store binary data in Python 2 byte
strings can't use Python 2 pickles from Python 3. It also means that
only ascii data can be unpickled.
A custom pickler / unpickler is suggested as the solution in this issue.
All the best,
Michael Foord
I use the encoding "latin1" for transparent byte/unicode conversion (do
not touch the values!).
This seems to be a bug in Python 3.2. Any suggestion?.
PS: The bytestream is protocol 2.
PPS: If there is consensus that this is a real bug, I would create an
issue in the tracker and try to get a minimal testcase.
- --
Jesus Cea Avion _/_/ _/_/_/ _/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/
jabber / xmpp:j...@jabber.org _/_/ _/_/ _/_/_/_/_/
. _/_/ _/_/ _/_/ _/_/ _/_/
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTWOomplgi5GaxT1NAQLTYAP/U+eqhQ5nIJyBAqSgYwPmkH4DOlMj4JnH
Jgt6okvOV0hRIXlZ7kbWI2l9OuQyUM4gAeTNDSjFaKs9Hswy26Ro6xhtjidivXDS
TKw6ocRx92/eHvgsOdEZjrE0D8l0dOqodZddbXELp2DjpYs9aozzAsjTHqNZDE1L
fujeTOhtUKw=
=/bcO
-----END PGP SIGNATURE-----
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
--
http://www.voidspace.org.uk/
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
