On Mon, Jan 9, 2012 at 2:04 PM, Brandon Wirtz <[email protected]> wrote: > I don’t use numpy, so I have no clue, but doesn’t Savez just compress save?
No, it uses a fairly different code path. > Can the same thing be achieved using Zip compress on the Save? I'm not 100% sure - save and savez end up in different C code and I didn't spend enough time examining the code to see how they are different. > Also could the GAE version of Numpy not be modified easily to not use a temp > file and just doing the compression in memory? Even if modifying it were easy (and I'm not sure about that), we'd prefer to not have a set of patches lying around that we have to apply to various third-party libraries every time that we upgrade them. Cheers, Brian > > > > > > From: [email protected] > [mailto:[email protected]] On Behalf Of Brian Quinlan > Sent: Sunday, January 08, 2012 6:43 PM > To: [email protected] > Subject: Re: [google-appengine] Re: function save from numpy works, but not > savez > > > > Hi, > > savez(), unlike save(), writes its intermediate results to a temporary file. > Since App Engine doesn't support temporary files, this doesn't work. > > > > Working around this should be fairly easy but you may have to careful that > your intermediate data isn't bigger than the available instance memory. > > > > Cheers, > > Brian > > > > On Mon, Jan 9, 2012 at 7:36 AM, redjava <[email protected]> wrote: > > savez(StringIO,numpy array) > it works for save(StringIO,numpy array), but will raise that error if > change save() to savez() > > Thanks, > redjava > > On Jan 7, 9:37 pm, Brian Quinlan <[email protected]> wrote: > >> On Sun, Jan 8, 2012 at 4:19 PM, redjava <[email protected]> wrote: >> > The rest is just a function call to savez >> >> With what arguments? >> >> Cheers, >> Brian >> >> >> >> >> >> >> >> >> >> > On Jan 7, 4:41 pm, Brian Quinlan <[email protected]> wrote: >> > > Are you sure that is the full traceback? >> >> > > On Sun, Jan 8, 2012 at 10:52 AM, redjava <[email protected]> wrote: >> > > > Hi, >> >> > > > The traceback for using savez is: >> > > > File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 491, >> > > > in savez >> > > > File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 543, >> > > > in _savez >> > > > File "C:\Program Files >> > > > (x86)\Google\google_appengine\google\appengine >> > > > \dist\tempfile.py", line 61, in PlaceHolder >> > > > raise NotImplementedError("Only tempfile.TemporaryFile is >> > > > available for use") >> > > > NotImplementedError: Only tempfile.TemporaryFile is available for >> > > > use >> >> > > > Thanks! >> > > > redjava >> >> > > > On Jan 6, 5:51 pm, Brian Quinlan <[email protected]> wrote: >> > > > > Hi, >> >> > > > > On Sat, Jan 7, 2012 at 12:43 PM, redjava <[email protected]> >> > > > > wrote: >> > > > > > To save large numpy array, one way is to create a StringIO, call >> > > > > > save() to compress the array and save it as Blob >> > > > > > This method works fine with save(), but not with savez(). >> > > > > > Instead >> > it >> > > > > > will raise "NotImplementedError: Only tempfile.TemporaryFile is >> > > > > > available for use" >> >> > > > > Could you provide the entire traceback? >> >> > > > > Cheers, >> > > > > Brian >> >> > > > > > -- >> > > > > > You received this message because you are subscribed to the >> > > > > > Google >> > > > Groups >> > > > > > "Google App Engine" group. >> > > > > > To post to this group, send email to >> > [email protected] >> > > > . >> > > > > > To unsubscribe from this group, send email to >> > > > > > [email protected]. >> > > > > > For more options, visit this group at >> > > > > >http://groups.google.com/group/google-appengine?hl=en. >> >> > > > -- >> > > > You received this message because you are subscribed to the Google >> > Groups >> > > > "Google App Engine" group. >> > > > To post to this group, send email to >> > > > [email protected] >> > . >> > > > To unsubscribe from this group, send email to >> > > > [email protected]. >> > > > For more options, visit this group at >> > > >http://groups.google.com/group/google-appengine?hl=en. >> >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Google App Engine" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group at >> >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
