Re: [Numpy-discussion] memmory management question
On Mon, Oct 28, 2013 at 6:25 AM, Pierre Haessig wrote: > > a.base > > In this document, it is mentionned several time that slicing yields > "views of the original data", but the .base attribute is not mentionned. > Should it be or is it out-of-scope of the Indexing guide ? > Indeed, that is not the least bit "basic" numpy usage -- the "basic way to do what you want is to keep the original array around in the first place -- i.e. give the slice a different name. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] memmory management question
On Mon, Oct 28, 2013 at 1:25 PM, Pierre Haessig wrote: > > Le 28/10/2013 13:40, Robert Kern a écrit : > > > What I didn't find (by quick googling) is how to access the original > > > array. Is it possible to access it (with Python code) ? > > > > a.base > Thanks! Is there a specific paragraph I missed in the user guide ? > > I had googled "numpy access original array" and first result is > http://docs.scipy.org/doc/numpy/user/basics.indexing.html > In this document, it is mentionned several time that slicing yields > "views of the original data", but the .base attribute is not mentionned. > Should it be or is it out-of-scope of the Indexing guide ? I think it's out-of-scope for "Numpy basics", certainly. It's mostly an implementation detail. There isn't much that you can usefully do with it as a user. http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.base.html -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] memmory management question
Le 28/10/2013 13:40, Robert Kern a écrit : > > What I didn't find (by quick googling) is how to access the original > > array. Is it possible to access it (with Python code) ? > > a.base Thanks! Is there a specific paragraph I missed in the user guide ? I had googled "numpy access original array" and first result is http://docs.scipy.org/doc/numpy/user/basics.indexing.html In this document, it is mentionned several time that slicing yields "views of the original data", but the .base attribute is not mentionned. Should it be or is it out-of-scope of the Indexing guide ? best, Pierre signature.asc Description: OpenPGP digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] memmory management question
On Mon, Oct 28, 2013 at 12:37 PM, Pierre Haessig wrote: > What I didn't find (by quick googling) is how to access the original > array. Is it possible to access it (with Python code) ? a.base -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] memmory management question
Hi, Along the line of what David said, I just looked at the flags : a = np.arange(10) a.flags [...] OWNDATA : True a = a[:3] a.flags [...] OWNDATA : False Indeed, after a=a[:3], a is not the same Python object but still points to the data of the first object. What I didn't find (by quick googling) is how to access the original array. Is it possible to access it (with Python code) ? best, Pierre signature.asc Description: OpenPGP digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] memmory management question
On 28 October 2013 03:13, Georgios Exarchakis wrote: > If yes then how do you release memorry by slicing away parts of an array? An array is a single Python object. In your example, there is always one reference pointing to the array (either the whole array or only a view), so the memory cannot be released. In your case, a = a[:1].copy() releases the memory by creating a new object and pointing a to this new one, so the big array gets unreferenced and thus, garbage collected. The price to pay is that, for some time, you have living in memory both the original array and your new one: if they were very big, you could run out of memory; and that you have to perform a memcopy (again, slow if you have lots of data). /David. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] memmory management question
Hi, I am using numpy with ipython from anaconda and I observe the following behavior: Python 2.7.5 |Anaconda 1.7.0 (64-bit)| (default, Jun 28 2013, 22:10:09) Type "copyright", "credits" or "license" for more information. IPython 1.0.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. Using matplotlib backend: Qt4Agg In [1]: a= np.random.rand(50,1000) In [2]: a = a[:1] In [3]: c= np.random.rand(50,1000) In [4]: After In[1] I have an extra 3.7 GB of memory used, but this memory is not released at In[2]. I thought there might be some clever memory management trick so I executted In[3] but that just added an extra 3.7GB of memorry without releasing anything. Is that the right behavior in this case? If yes then how do you release memorry by slicing away parts of an array? Can you give me a description of the numpy internals in this case? Thank you very much for your time, Georgios ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
