On Mon, Jan 17, 2011 at 06:35, Tom Holderness <[email protected]> wrote: > Hi, > > How do I find the maximum possible array size for a given data type on a > given architecture? > For example if I do the following on a 32-bit Windows machine: > > matrix = np.zeros((8873,9400),np.dtype('f8')) > > I get, > Traceback (most recent call last): > File "<pyshell#115>", line 1, in <module> > matrix = np.zeros((8873,9400),np.dtype('f8')) > MemoryError > > If I reduce the matrix size then it works. > However, if I run the original command on an equivalent 32-bit Linux machine > this works fine (presumably some limit of memory allocation in the Windows > kernel? I tested increasing the available RAM and it doesn't solve the > problem). > > Is there a way I can find this limit? When distributing software to users > (who all run different architectures) it would be great if we could check > this before running the process and catch the error before the user hits > "run".
No, there is no way to know a priori. It's not just dependent on the system, but also on where memory is currently allocated. You could in principle determine the maximum size of the address space from the system. However, if you have, say, 3 Gb of address space for your process, and you allocate two blocks of 1 Gb each, those allocations may not be right next to each other. If the blocks start on 0.5 Gb and 2.0 Gb respectively, there is 1 Gb of free address space, but broken up into two 0.5 Gb blocks. You could not allocate a third 1 Gb block because there is nowhere to put it. You can detect this problem early by doing an np.empty() of the right size early in the program. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
