Re: [Numpy-discussion] Create a n-D grid; meshgrid alternative
On Sun, May 10, 2015 at 4:40 AM, Stefan Otte stefan.o...@gmail.com wrote: Hey, quite often I want to evaluate a function on a grid in a n-D space. What I end up doing (and what I really dislike) looks something like this: x = np.linspace(0, 5, 20) M1, M2 = np.meshgrid(x, x) X = np.column_stack([M1.flatten(), M2.flatten()]) X.shape # (400, 2) fancy_function(X) I don't think I ever used `meshgrid` in any other way. Is there a better way to create such a grid space? I feel like our house style has moved away from automatic flattening, and would maybe we should be nudging people towards something more like # using proposed np.stack from pull request #5605 X = np.stack(np.meshgrid(x, x), axis=-1) assert X.shape == (20, 20, 2) fancy_function(X) # vectorized to accept any array with shape (..., 2) -n -- Nathaniel J. Smith -- http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Create a n-D grid; meshgrid alternative
On Sun, May 10, 2015 at 4:40 AM, Stefan Otte stefan.o...@gmail.com wrote: Hey, quite often I want to evaluate a function on a grid in a n-D space. What I end up doing (and what I really dislike) looks something like this: x = np.linspace(0, 5, 20) M1, M2 = np.meshgrid(x, x) X = np.column_stack([M1.flatten(), M2.flatten()]) X.shape # (400, 2) fancy_function(X) I don't think I ever used `meshgrid` in any other way. Is there a better way to create such a grid space? I wrote myself a little helper function: def gridspace(linspaces): return np.column_stack([space.flatten() for space in np.meshgrid(*linspaces)]) But maybe something like this should be part of numpy? Isn't what you are trying to build a cartesian product function? There is a neat, efficient implementation of such a function in StackOverflow, by our own pv.: http://stackoverflow.com/questions/1208118/using-numpy-to-build-an-array-of-all-combinations-of-two-arrays/1235363#1235363 Perhaps we could make this part of numpy.lib.arraysetops? Isthere room for other combinatoric generators, i.e. combinations, permutations... as in itertools? Jaime -- (\__/) ( O.o) ( ) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Create a n-D grid; meshgrid alternative
On 2015-05-10 14:46:12, Jaime Fernández del Río jaime.f...@gmail.com wrote: Isn't what you are trying to build a cartesian product function? There is a neat, efficient implementation of such a function in StackOverflow, by our own pv.: http://stackoverflow.com/questions/1208118/using-numpy-to-build-an-array-of-all-combinations-of-two-arrays/1235363#1235363 And a slightly faster version just down that page ;) Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Generalize hstack/vstack -- stack; Block matrices like in matlab
Hey, Just a quick update. I updated the pull request and renamed `stack` into `block`. Have a look: https://github.com/numpy/numpy/pull/5057 I'm sticking with simple initial implementation because it's simple and does what you think it does. Cheers, Stefan On Fri, Oct 31, 2014 at 2:13 PM Stefan Otte stefan.o...@gmail.com wrote: To make the last point more concrete the implementation could look something like this (note that I didn't test it and that it still takes some work): def bmat(obj, ldict=None, gdict=None): return matrix(stack(obj, ldict, gdict)) def stack(obj, ldict=None, gdict=None): # the old bmat code minus the matrix calls if isinstance(obj, str): if gdict is None: # get previous frame frame = sys._getframe().f_back glob_dict = frame.f_globals loc_dict = frame.f_locals else: glob_dict = gdict loc_dict = ldict return _from_string(obj, glob_dict, loc_dict) if isinstance(obj, (tuple, list)): # [[A,B],[C,D]] arr_rows = [] for row in obj: if isinstance(row, N.ndarray): # not 2-d return concatenate(obj, axis=-1) else: arr_rows.append(concatenate(row, axis=-1)) return concatenate(arr_rows, axis=0) if isinstance(obj, N.ndarray): return obj I basically turned the old `bmat` into `stack` and removed the matrix calls. Best, Stefan On Wed, Oct 29, 2014 at 3:59 PM, Stefan Otte stefan.o...@gmail.com wrote: Hey, there are several ways how to proceed. - My proposed solution covers the 80% case quite well (at least I use it all the time). I'd convert the doctests into unittests and we're done. - We could slightly change the interface to leave out the surrounding square brackets, i.e. turning `stack([[a, b], [c, d]])` into `stack([a, b], [c, d])` - We could extend it even further allowing a filler value for non set values and a shape argument. This could be done later as well. - `bmat` is not really matrix specific. We could refactor `bmat` a bit to use the same logic in `stack`. Except the `matrix` calls `bmat` and `_from_string` are pretty agnostic to the input. I'm in favor of the first or last approach. The first: because it already works and is quite simple. The last: because the logic and tests of both `bmat` and `stack` would be the same and the feature to specify a string representation of the block matrix is nice. Best, Stefan On Tue, Oct 28, 2014 at 7:46 PM, Nathaniel Smith n...@pobox.com wrote: On 28 Oct 2014 18:34, Stefan Otte stefan.o...@gmail.com wrote: Hey, In the last weeks I tested `np.asarray(np.bmat())` as `stack` function and it works quite well. So the question persits: If `bmat` already offers something like `stack` should we even bother implementing `stack`? More code leads to more bugs and maintenance work. (However, the current implementation is only 5 lines and by using `bmat` which would reduce that even more.) In the long run we're trying to reduce usage of np.matrix and ideally deprecate it entirely. So yes, providing ndarray equivalents of matrix functionality (like bmat) is valuable. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Create a n-D grid; meshgrid alternative
I just drafted different versions of the `gridspace` function: https://tmp23.tmpnb.org/user/1waoqQ8PJBJ7/notebooks/2015-05%20gridspace.ipynb Beste Grüße, Stefan On Sun, May 10, 2015 at 1:40 PM, Stefan Otte stefan.o...@gmail.com wrote: Hey, quite often I want to evaluate a function on a grid in a n-D space. What I end up doing (and what I really dislike) looks something like this: x = np.linspace(0, 5, 20) M1, M2 = np.meshgrid(x, x) X = np.column_stack([M1.flatten(), M2.flatten()]) X.shape # (400, 2) fancy_function(X) I don't think I ever used `meshgrid` in any other way. Is there a better way to create such a grid space? I wrote myself a little helper function: def gridspace(linspaces): return np.column_stack([space.flatten() for space in np.meshgrid(*linspaces)]) But maybe something like this should be part of numpy? Best, Stefan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Create a n-D grid; meshgrid alternative
On Sun, May 10, 2015 at 7:05 AM, Stefan Otte stefan.o...@gmail.com wrote: I just drafted different versions of the `gridspace` function: https://tmp23.tmpnb.org/user/1waoqQ8PJBJ7/notebooks/2015-05%20gridspace.ipynb The link seems to be broken... Jaime -- (\__/) ( O.o) ( ) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] read not byte aligned records
For the archive, I tried to use bitarray instead of bitstring and for same file parsing went from 180ms to 60ms. Code was finally shorter and more simple but less easy to jump into (documentation). Performance is still far from using fromstring or fromfile which gives like 5ms for similar size of file but byte aligned. Aymeric my code is below: def readBitarray(self, bita, channelList=None): reads stream of record bytes using bitarray module needed for not byte aligned data Parameters bitarray : stream stream of bytes channelList : List of str, optional Returns rec : numpy recarray contains a matrix of raw data in a recarray (attributes corresponding to channel name) from bitarray import bitarray B = bitarray(endian=little) # little endian by default B.frombytes(bytes(bita)) # initialise data structure if channelList is None: channelList = self.channelNames format = [] for channel in self: if channel.name in channelList: format.append(channel.RecordFormat) buf = recarray(self.numberOfRecords, format) # read data for chan in range(len(self)): if self[chan].name in channelList: record_bit_size = self.CGrecordLength * 8 temp = [B[self[chan].posBitBeg + record_bit_size * i:\ self[chan].posBitEnd + record_bit_size * i]\ for i in range(self.numberOfRecords)] nbytes = len(temp[0].tobytes()) if not nbytes == self[chan].nBytes and \ self[chan].signalDataType not in (6, 7, 8, 9, 10, 11, 12): # not Ctype byte length byte = 8 * (self[chan].nBytes - nbytes) * bitarray([False]) for i in range(self.numberOfRecords): # extend data of bytes to match numpy requirement temp[i].append(byte) temp = [self[chan].CFormat.unpack(temp[i].tobytes())[0] \ for i in range(self.numberOfRecords)] buf[self[chan].name] = asarray(temp) return buf Le 05/05/15 15:39, Benjamin Root a écrit : I have been very happy with the bitarray package. I don't know if it is faster than bitstring, but it is worth a mention. Just watch out for any hashing operations on its objects, it doesn't seem to do them right (set(), dict(), etc...), but comparison operations work just fine. Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
