Re: numpy.memmap advice?

2009-02-19 Thread Lionel
On Feb 18, 12:35 pm, Carl Banks pavlovevide...@gmail.com wrote: On Feb 18, 10:48 am, Lionel lionel.ke...@gmail.com wrote: Thanks Carl, I like your solution. Am I correct in my understanding that memory is allocated at the slicing step in your example i.e. when reshaped_data is sliced using

Re: numpy.memmap advice?

2009-02-19 Thread Carl Banks
On Feb 19, 9:34 am, Lionel lionel.ke...@gmail.com wrote: On Feb 18, 12:35 pm, Carl Banks pavlovevide...@gmail.com wrote: On Feb 18, 10:48 am, Lionel lionel.ke...@gmail.com wrote: Thanks Carl, I like your solution. Am I correct in my understanding that memory is allocated at the

Re: numpy.memmap advice?

2009-02-19 Thread sturlamolden
On 19 Feb, 03:13, Carl Banks pavlovevide...@gmail.com wrote: The offset parameter of mmap itself would be useful to map small portions of gigabyte-sized files, and maybe numpy.memmap can take advantage of that if the user passes an offset parameter.   NumPy's memmap is just a wrapper for

Re: numpy.memmap advice?

2009-02-19 Thread Carl Banks
On Feb 19, 10:00 am, sturlamolden sturlamol...@yahoo.no wrote: On 19 Feb, 03:13, Carl Banks pavlovevide...@gmail.com wrote: The offset parameter of mmap itself would be useful to map small portions of gigabyte-sized files, and maybe numpy.memmap can take advantage of that if the user

Re: numpy.memmap advice?

2009-02-19 Thread Lionel
On Feb 19, 9:51 am, Carl Banks pavlovevide...@gmail.com wrote: On Feb 19, 9:34 am, Lionel lionel.ke...@gmail.com wrote: On Feb 18, 12:35 pm, Carl Banks pavlovevide...@gmail.com wrote: On Feb 18, 10:48 am, Lionel lionel.ke...@gmail.com wrote: Thanks Carl, I like your solution. Am I

Re: numpy.memmap advice?

2009-02-19 Thread Carl Banks
On Feb 19, 10:36 am, Lionel lionel.ke...@gmail.com wrote: On Feb 19, 9:51 am, Carl Banks pavlovevide...@gmail.com wrote: On Feb 19, 9:34 am, Lionel lionel.ke...@gmail.com wrote: On Feb 18, 12:35 pm, Carl Banks pavlovevide...@gmail.com wrote: On Feb 18, 10:48 am, Lionel

Re: numpy.memmap advice?

2009-02-19 Thread Lionel
On Feb 19, 12:26 pm, Carl Banks pavlovevide...@gmail.com wrote: On Feb 19, 10:36 am, Lionel lionel.ke...@gmail.com wrote: On Feb 19, 9:51 am, Carl Banks pavlovevide...@gmail.com wrote: On Feb 19, 9:34 am, Lionel lionel.ke...@gmail.com wrote: On Feb 18, 12:35 pm, Carl Banks

Re: numpy.memmap advice?

2009-02-18 Thread Carl Banks
On Feb 17, 3:08 pm, Lionel lionel.ke...@gmail.com wrote: Hello all, On a previous thread (http://groups.google.com/group/comp.lang.python/ browse_thread/thread/64da35b811e8f69d/67fa3185798ddd12? hl=enlnk=gstq=keene#67fa3185798ddd12) I was asking about reading in binary data. Briefly, my data

Re: numpy.memmap advice?

2009-02-18 Thread Lionel
On Feb 18, 12:56 am, Carl Banks pavlovevide...@gmail.com wrote: On Feb 17, 3:08 pm, Lionel lionel.ke...@gmail.com wrote: Hello all, On a previous thread (http://groups.google.com/group/comp.lang.python/ browse_thread/thread/64da35b811e8f69d/67fa3185798ddd12?

Re: numpy.memmap advice?

2009-02-18 Thread Carl Banks
On Feb 18, 10:48 am, Lionel lionel.ke...@gmail.com wrote: Thanks Carl, I like your solution. Am I correct in my understanding that memory is allocated at the slicing step in your example i.e. when reshaped_data is sliced using interesting_data = reshaped_data[:, 50:100]? In other words, given

Re: numpy.memmap advice?

2009-02-18 Thread sturlamolden
On 18 Feb, 00:08, Lionel lionel.ke...@gmail.com wrote: 1) What is recarray? An ndarray of what C programmers know as a struct, in which each field is accessible by its name. That is, struct rgba{ unsigned char r; unsigned char g; unsigned char b; unsigned char a; }; struct rgba

Re: numpy.memmap advice?

2009-02-18 Thread Carl Banks
On Feb 18, 4:23 pm, sturlamolden sturlamol...@yahoo.no wrote: On 18 Feb, 00:08, Lionel lionel.ke...@gmail.com wrote: 1) What is recarray? An ndarray of what C programmers know as a struct, in which each field is accessible by its name. That is, struct rgba{   unsigned char r;  

numpy.memmap advice?

2009-02-17 Thread Lionel
Hello all, On a previous thread (http://groups.google.com/group/comp.lang.python/ browse_thread/thread/64da35b811e8f69d/67fa3185798ddd12? hl=enlnk=gstq=keene#67fa3185798ddd12) I was asking about reading in binary data. Briefly, my data consists of complex numbers, 32-bit floats for real and

Re: numpy.memmap advice?

2009-02-17 Thread Robert Kern
On 2009-02-17 17:08, Lionel wrote: Hello all, On a previous thread (http://groups.google.com/group/comp.lang.python/ browse_thread/thread/64da35b811e8f69d/67fa3185798ddd12? hl=enlnk=gstq=keene#67fa3185798ddd12) I was asking about reading in binary data. Briefly, my data consists of complex