[Numpy-discussion] What to use to read and write numpy arrays to a file?
In looking for simple ways to read and write data (in a text readable format) to and from a file and later restoring the actual data when reading back in, I've found that numpy arrays don't seem to play well with repr and eval. E.g. to write some data (mixed types) to a file I can do this (fp is an open file), thedata=[3.0,-4.9+2.0j,'another string'] repvars= repr(thedata)+\n fp.write(repvars) Then to read it back and restore the data each to its original type, strvars= fp.readline() sonofdata= eval(strvars) which gives back the original data list. BUT when I try this with numpy arrays in the data list I find that repr of an array adds extra end-of-lines and that messes up the simple restoration of the data using eval. Am I missing something simple? I know I've seen people recommend ways to save arrays to files, but I'm wondering what is the most straight-forward? I really like the simple, pythonic approach of the repr - eval pairing. Thanks for any advice. (yes, I am googling, too) -- Lou Pecora, my views are my own. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What to use to read and write numpy arrays to a file?
Hi, The repr - eval pair does not work with numpy. You can simply do a tofile() from file(). Matthieu 2008/12/8 Lou Pecora [EMAIL PROTECTED]: In looking for simple ways to read and write data (in a text readable format) to and from a file and later restoring the actual data when reading back in, I've found that numpy arrays don't seem to play well with repr and eval. E.g. to write some data (mixed types) to a file I can do this (fp is an open file), thedata=[3.0,-4.9+2.0j,'another string'] repvars= repr(thedata)+\n fp.write(repvars) Then to read it back and restore the data each to its original type, strvars= fp.readline() sonofdata= eval(strvars) which gives back the original data list. BUT when I try this with numpy arrays in the data list I find that repr of an array adds extra end-of-lines and that messes up the simple restoration of the data using eval. Am I missing something simple? I know I've seen people recommend ways to save arrays to files, but I'm wondering what is the most straight-forward? I really like the simple, pythonic approach of the repr - eval pairing. Thanks for any advice. (yes, I am googling, too) -- Lou Pecora, my views are my own. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What to use to read and write numpy arrays to a file?
On Mon, Dec 8, 2008 at 14:54, Lou Pecora [EMAIL PROTECTED] wrote: In looking for simple ways to read and write data (in a text readable format) to and from a file and later restoring the actual data when reading back in, I've found that numpy arrays don't seem to play well with repr and eval. E.g. to write some data (mixed types) to a file I can do this (fp is an open file), thedata=[3.0,-4.9+2.0j,'another string'] repvars= repr(thedata)+\n fp.write(repvars) Then to read it back and restore the data each to its original type, strvars= fp.readline() sonofdata= eval(strvars) which gives back the original data list. BUT when I try this with numpy arrays in the data list I find that repr of an array adds extra end-of-lines and that messes up the simple restoration of the data using eval. I don't see any extra end-of-lines. Are you sure you aren't talking about the ... when you are saving large arrays? You will need to use set_printoptions() to disable that (threshold=sys.maxint). You should also adjust use precision=18, suppress=False. That should mostly work, but it's never a certain thing. Am I missing something simple? I know I've seen people recommend ways to save arrays to files, but I'm wondering what is the most straight-forward? I really like the simple, pythonic approach of the repr - eval pairing. Thanks for any advice. (yes, I am googling, too) The most bulletproof way would be to use numpy.save() and numpy.load(), but this is a binary format, not a text one. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What to use to read and write numpy arrays to a file?
--- On Mon, 12/8/08, Matthieu Brucher [EMAIL PROTECTED] wrote: From: Matthieu Brucher [EMAIL PROTECTED] Subject: Re: [Numpy-discussion] What to use to read and write numpy arrays to a file? To: Discussion of Numerical Python numpy-discussion@scipy.org Date: Monday, December 8, 2008, 3:56 PM Hi, The repr - eval pair does not work with numpy. You can simply do a tofile() from file(). Matthieu Yes, I found the tofile/fromfile pair, but they don't preserve the shape. Sorry, I should have been clearer on that in my request. I will be saving arrays whose shape I may not know later when I read them in. I'd like that information to be preserved. Thanks. -- Lou Pecora, my views are my own. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What to use to read and write numpy arrays to a file?
--- On Mon, 12/8/08, Robert Kern [EMAIL PROTECTED] wrote: From: Robert Kern [EMAIL PROTECTED] Subject: Re: [Numpy-discussion] What to use to read and write numpy arrays to a file? The most bulletproof way would be to use numpy.save() and numpy.load(), but this is a binary format, not a text one. -- Robert Kern Thanks, Robert. I may have to go that route, assuming that the save and load pair preserve shape, i.e. I don't have to know the shape when I read back in. -- Lou Pecora, my views are my own. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What to use to read and write numpy arrays to a file?
On Mon, Dec 8, 2008 at 15:26, Lou Pecora [EMAIL PROTECTED] wrote: --- On Mon, 12/8/08, Robert Kern [EMAIL PROTECTED] wrote: From: Robert Kern [EMAIL PROTECTED] Subject: Re: [Numpy-discussion] What to use to read and write numpy arrays to a file? The most bulletproof way would be to use numpy.save() and numpy.load(), but this is a binary format, not a text one. -- Robert Kern Thanks, Robert. I may have to go that route, assuming that the save and load pair preserve shape, i.e. I don't have to know the shape when I read back in. They do. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
