Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-07 Thread Sebastian Berg
On Sa, 2015-03-07 at 10:23 +, Robert Kern wrote: > On Sat, Mar 7, 2015 at 9:54 AM, Pauli Virtanen wrote: > > > How about an extra use_pickle=True kwarg that can be used to disable > > using pickle altogether in these routines? > > If we do, I'd vastly prefer `forbid_pickle=False`. The use_pi

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-07 Thread Robert Kern
On Sat, Mar 7, 2015 at 9:54 AM, Pauli Virtanen wrote: > How about an extra use_pickle=True kwarg that can be used to disable > using pickle altogether in these routines? If we do, I'd vastly prefer `forbid_pickle=False`. The use_pickle spelling suggests that you are asking it to use pickle when

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-07 Thread Pauli Virtanen
07.03.2015, 01:29, Julian Taylor kirjoitti: > On 07.03.2015 00:20, Pauli Virtanen wrote: >> 06.03.2015, 22:43, Eric Firing kirjoitti: >>> On 2015/03/06 10:23 AM, Pauli Virtanen wrote: 06.03.2015, 20:00, Benjamin Root kirjoitti: > A slightly different way to look at this is one of sharing d

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Eric Firing
On 2015/03/06 1:29 PM, Julian Taylor wrote: > I think the ship for a warning has long sailed. At this point its > probably more an annoyance for python3 users and will not prevent many > more python2 users from saving files that can't be loaded into python3. The point of a warning is that anything

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Julian Taylor
On 07.03.2015 00:20, Pauli Virtanen wrote: > 06.03.2015, 22:43, Eric Firing kirjoitti: >> On 2015/03/06 10:23 AM, Pauli Virtanen wrote: >>> 06.03.2015, 20:00, Benjamin Root kirjoitti: A slightly different way to look at this is one of sharing data. If I am working on a system with 3.4 and

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Pauli Virtanen
06.03.2015, 22:23, Pauli Virtanen kirjoitti: > 06.03.2015, 20:00, Benjamin Root kirjoitti: >> A slightly different way to look at this is one of sharing data. If I am >> working on a system with 3.4 and I want to share data with others who may >> be using a mix of 2.7 and 3.3 systems, this problem

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Pauli Virtanen
06.03.2015, 22:43, Eric Firing kirjoitti: > On 2015/03/06 10:23 AM, Pauli Virtanen wrote: >> 06.03.2015, 20:00, Benjamin Root kirjoitti: >>> A slightly different way to look at this is one of sharing data. If I am >>> working on a system with 3.4 and I want to share data with others who may >>> be

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Eric Firing
On 2015/03/06 10:23 AM, Pauli Virtanen wrote: > 06.03.2015, 20:00, Benjamin Root kirjoitti: >> A slightly different way to look at this is one of sharing data. If I am >> working on a system with 3.4 and I want to share data with others who may >> be using a mix of 2.7 and 3.3 systems, this problem

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Pauli Virtanen
06.03.2015, 20:00, Benjamin Root kirjoitti: > A slightly different way to look at this is one of sharing data. If I am > working on a system with 3.4 and I want to share data with others who may > be using a mix of 2.7 and 3.3 systems, this problem makes npz format much > less attractive. pickle i

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Benjamin Root
A slightly different way to look at this is one of sharing data. If I am working on a system with 3.4 and I want to share data with others who may be using a mix of 2.7 and 3.3 systems, this problem makes npz format much less attractive. Ben Root On Fri, Mar 6, 2015 at 12:51 PM, Charles R Harris

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Charles R Harris
On Fri, Mar 6, 2015 at 10:34 AM, Sebastian wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi all, > > As this also affects .npy files, which uses pickle internally, why can't > this be done by Numpy itself? This breaks backwards compatibility in a > very bad way in my opinion. >

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Sebastian
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi all, As this also affects .npy files, which uses pickle internally, why can't this be done by Numpy itself? This breaks backwards compatibility in a very bad way in my opinion. The company I worked for uses Numpy and consorts a lot and also has

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Ryan Nelson
Arnd, I can see where this is an issue. If you are trying to update your code for Py3, I still think that it would really help to add a version attribute of some sort to your new HDF files. You can then write a little check in your access code that looks for this variable. If it is not present, yo

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Arnd Baecker
On Fri, 6 Mar 2015, Pauli Virtanen wrote: > Arnd Baecker web.de> writes: > [clip] >> Still I would have thought that this should be working out-of-the box, >> i.e. without the pickle.loads trick? > > Pickle files should be considered incompatible between Python 2 and Python 3. > > Python 3 interp

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-06 Thread Pauli Virtanen
Arnd Baecker web.de> writes: [clip] > Still I would have thought that this should be working out-of-the box, > i.e. without the pickle.loads trick? Pickle files should be considered incompatible between Python 2 and Python 3. Python 3 interprets all bytes objects saved by Python 2 as str and att

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-05 Thread Arnd Baecker
On Thu, 5 Mar 2015, Ryan Nelson wrote: This works if run from Py3. Don't know if it will *always* work. From that GH discussion you linked, it sounds like that is a bit of a hack. Great - based on your code I could modify my loader routine so that on python 3 it can load the files generated o

Re: [Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-05 Thread Ryan Nelson
This works if run from Py3. Don't know if it will *always* work. From that GH discussion you linked, it sounds like that is a bit of a hack. ## """Illustrate problem with pytables data - python 2 to python 3.""" from __future__ import print_function import sys import numpy as np impor

[Numpy-discussion] numpy pickling problem - python 2 vs. python 3

2015-03-04 Thread Anrd Baecker
Dear all, when preparing the transition of our repositories from python 2 to python 3, I encountered a problem loading pytables (.h5) files generated using python 2. I suspect that it is caused by a problem with pickling numpy arrays under python 3: The code appended at the end of this mail works