Re: [Numpy-discussion] cumsum much slower than simple loop?
10.02.2012 05:39, Dave Cook kirjoitti: Why is numpy.cumsum (along axis=0) so much slower than a simple loop? The same goes for numpy.add.accumulate The reason is loop ordering. The reduction operator when using `cumsum` or `add.reduce` does the summation in the inmost loop, whereas the `loopcumsum` has the summation in the outmost loop. Although both algorithms do the same number of operations, the latter is more efficient with regards to CPU cache (and maybe memory data dependency) --- the arrays are in C-order so summing along the first axis is wasteful as the elements are far from each other in memory. The effect goes away, if you use a Fortran-ordered array: a = np.array(a, order='F') print a.shape Numpy does not currently have heuristics to determine when swapping the loop order would be beneficial in accumulation and reductions. It does, however, have the heuristics in place for elementwise operations. -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] On making Numpy 1.7 a long term support release.
On Sun, Feb 5, 2012 at 7:19 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sun, Feb 5, 2012 at 7:33 AM, Travis Oliphant tra...@continuum.io wrote: I think supporting Python 2.5 and above is completely fine. I'd even be in favor of bumping up to Python 2.6 for NumPy 1.7 and certainly for NumPy 2.8 +1 for dropping Python 2.5 support also for an LTS release. That will make it a lot easier to use str.format() and the with statement (plus many other things) going forward, without having to think about if your changes can be backported to that LTS release. At the risk of sounding like a broken record, I would really like to stay to 2.4, especially for a long term release :) This is still the basis used by a lots of long-term python products. If we can support 2.4 for a LTS, I would then be much more comfortable to allow bumping to 2.5 for 1.8. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] On making Numpy 1.7 a long term support release.
On 5 February 2012 07:19, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sun, Feb 5, 2012 at 7:33 AM, Travis Oliphant tra...@continuum.io wrote: I think supporting Python 2.5 and above is completely fine. I'd even be in favor of bumping up to Python 2.6 for NumPy 1.7 and certainly for NumPy 2.8 +1 for dropping Python 2.5 support also for an LTS release. That will make it a lot easier to use str.format() and the with statement (plus many other things) going forward, without having to think about if your changes can be backported to that LTS release. The with statement works just fine in python 2.5, all you have to do is 'from __future__ import with_statement'. As for str.format, well... Ralf On Feb 4, 2012, at 10:13 PM, Bruce Southey wrote: On Sat, Feb 4, 2012 at 6:07 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sat, Feb 4, 2012 at 3:03 PM, Travis Oliphant tra...@continuum.io wrote: We are spending a lot of time on NumPy and will be for the next few months. I think that 1.8 will be a better long term release. We need a few more fundamental features yet. Look for a roadmap document for discussion from Mark Wiebe and I within the week about NumPy 1.8 which has a target release of June 2012. Looking forward to that document. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion A suitable long term release would include deprecating old macros, datetime and einsum. While I would like to include NA, I am rather concerned with the recent bugs that have been uncovered with it. So I am rather wary of having to forced to backport fixes simply because someone said we would support with bug fixes for the next 2-3 years. Rather at least clearly indicate that not every fix will be backported. I propose that we use this opportunity end support for Python 2.4 especially since Red Hat Enterprise Linux (RHEL) 4 is February 29th, 2012. According to SourceForge, the last available binary release for Python 2.4 was for numpy 1.2.1 (released 2008-10-29). There is still quite a few downloads (3769) of the Python 2.5 numpy 1,6.1 binary. Bruce ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] simple manipulations of numpy arrays
Hi, I am relatively new to numpy and am seeking some advice on an appropriate way to do the following simple task. The idea is to build a class that will allow a user to easily remove and keep columns and rows in a 2D numpy array. An outline of the class is as follows: class DataEditor(object): def __init__(self, data_array): self.data_array = data_array self.edited_data_array = None def get_edited_data(self): return self.edited_data_array def remove_cols(self, a_list_of_column_numbers): remove the specified columns, but keep the rest # some functionality to produce self.edited_data_array def remove_rows(self, a_list_of_row_numbers): remove the specified rows, but keep the rest # some functionality to produce self.edited_data_array def keep_cols(self, a_list_of_column_numbers): keep the specified columns, but remove the rest # some functionality to produce self.edited_data_array def keep_rows(self, a_list_of_row_numbers): keep the specified rows, but remove the rest # some functionality to produce self.edited_data_array Usage would be something like the following: import numpy as np import data_editor data = np.random.rand(7,9) editor = data_editor.DataEditor(data) editor.remove_rows([2,4]) editor.keep_cols([3,5,7]) edited_data = data_editor.get_edited_array() I don't have much experience using them, but would using a masked array would make sense in this context? If so, how does one convert a masked array to a 'normal' array with only the unmasked values present from the original? Or, is another approach more appropriate. Thank you for your help. -Brad ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] simple manipulations of numpy arrays
This is not yet released (but will be in the near future): http://readthedocs.org/docs/astropy/en/latest/table/index.html https://github.com/astropy/astropy/blob/master/astropy/table/table.py You can at least use this as an example of how to add rows and columns to a structured array. Or be an early adopter and install and use astropy.table. :-) - Tom On Fri, Feb 10, 2012 at 9:29 AM, Brad Reisfeld brad.reisf...@gmail.com wrote: Hi, I am relatively new to numpy and am seeking some advice on an appropriate way to do the following simple task. The idea is to build a class that will allow a user to easily remove and keep columns and rows in a 2D numpy array. An outline of the class is as follows: class DataEditor(object): def __init__(self, data_array): self.data_array = data_array self.edited_data_array = None def get_edited_data(self): return self.edited_data_array def remove_cols(self, a_list_of_column_numbers): remove the specified columns, but keep the rest # some functionality to produce self.edited_data_array def remove_rows(self, a_list_of_row_numbers): remove the specified rows, but keep the rest # some functionality to produce self.edited_data_array def keep_cols(self, a_list_of_column_numbers): keep the specified columns, but remove the rest # some functionality to produce self.edited_data_array def keep_rows(self, a_list_of_row_numbers): keep the specified rows, but remove the rest # some functionality to produce self.edited_data_array Usage would be something like the following: import numpy as np import data_editor data = np.random.rand(7,9) editor = data_editor.DataEditor(data) editor.remove_rows([2,4]) editor.keep_cols([3,5,7]) edited_data = data_editor.get_edited_array() I don't have much experience using them, but would using a masked array would make sense in this context? If so, how does one convert a masked array to a 'normal' array with only the unmasked values present from the original? Or, is another approach more appropriate. Thank you for your help. -Brad ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] [ANN] new solver for multiobjective optimization problems
hi, I'm glad to inform you about new Python solver for multiobjective optimization (MOP). Some changes committed to solver interalg made it capable of handling global nonlinear constrained multiobjective problem (MOP), see the page for more details. Using interalg you can be 100% sure your result covers whole Pareto front according to the required tolerances on objective functions. Available features include real-time or final graphical output, possibility of involving parallel calculations, handling both continuous and discrete variables, export result to xls files. Regards, D. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] simple manipulations of numpy arrays
On Feb 10, 2012, at 3:29 PM, Brad Reisfeld wrote: Hi, I am relatively new to numpy and am seeking some advice on an appropriate way to do the following simple task. The idea is to build a class that will allow a user to easily remove and keep columns and rows in a 2D numpy array. clip Apart from the good suggestions already made, you may also find useful the carray package available in: https://github.com/FrancescAlted/carry It implements a ctable object that has different capabilities: * Allows addition and removal of columns very efficiently * Allows to enlarge and shrink the ctable * Supports compression (via the fast blosc compressor) * If numexpr is installed, you can seamlessly operate with columns efficiently * You can efficiently select rows using complex conditions (needs numexpr too) You can have a quick look at how this works in the second part of the tutorial: https://github.com/FrancescAlted/carray/blob/master/doc/tutorial.rst Hope it helps, -- Francesc Alted ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] simple manipulations of numpy arrays
On Feb 10, 2012, at 4:50 PM, Francesc Alted wrote: https://github.com/FrancescAlted/carry Hmm, this should be: https://github.com/FrancescAlted/carray Blame my (too) smart spell corrector. -- Francesc Alted ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [ANN] new solver for multiobjective optimization problems
And where do we find this gem? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Creating parallel curves
Hi All, my apologies for my deep ignorance about math stuff; I guess I should be able to find this out but I keep getting impossible results. Basically I have a set of x, y data (around 1,000 elements each) and I want to create 2 parallel curves (offset curves) to the original one; parallel means curves which are displaced from the base curve by a constant offset, either positive or negative, in the direction of the curve's normal. Something like this: http://pyx.sourceforge.net/examples/drawing2/parallel.html The point in the x variable are monotonically increasing. For every point in the curve, I also have the angle this point creates to the vertical (y) direction, so I naively thought of doing this: angles = numpy.deg2rad(angles) x_low = x - DISTANCE*numpy.cos(angles) y_low = y + DISTANCE*numpy.sin(angles) x_high = x + DISTANCE*numpy.cos(angles) y_high = y - DISTANCE*numpy.sin(angles) But by plotting these thing out with matplotlib it seems to me they don't really look very parallel nor very constant-distance. I admit the matplotlib axis scales can play a significant role here, but I was wondering if some of you could spot my dumb mistake (and maybe provide a way to nicely take into account the different scales of the matplotlib axes so that the curves really look parallel...). Thank you in advance for any suggestion. Andrea. Imagination Is The Only Weapon In The War Against Reality. http://xoomer.alice.it/infinity77/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Creating parallel curves
Andrea, Basically I have a set of x, y data (around 1,000 elements each) and I want to create 2 parallel curves (offset curves) to the original one; parallel means curves which are displaced from the base curve by a constant offset, either positive or negative, in the direction of the curve's normal. Something like this: http://pyx.sourceforge.net/examples/drawing2/parallel.html THis is called buffering in GIS parlance -- there are functions available to do it in GIS an computational geometry libraries: you might look in the shapely package: https://github.com/sgillies/shapely or CGAL http://www.cgal.org/ If the overhead of these packages is too much, and you still want to write your own code, try googling: buffering a line GIS algorithm or something like that, and you'll find pointers. But by plotting these thing out with matplotlib it seems to me they don't really look very parallel nor very constant-distance. as we say on the wxPython list -- post a fully functional example, so we can check it out. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] trapezoidal grids
Hello, Say I have four corner points a = (X0, Y0), b = (X1, Y1), c = (X2, Y2) and d = (X3, Y3): a--b \/ \ / cd Is there a function like meshgrid that would return me a grid of points linearly interpolating those four corner points? Thanks, ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] trapezoidal grids
On Fri, Feb 10, 2012 at 9:26 AM, Hugo Gagnon sourceforge.nu...@user.fastmail.fm wrote: Hello, Say I have four corner points a = (X0, Y0), b = (X1, Y1), c = (X2, Y2) and d = (X3, Y3): a--b \ / \ / cd Is there a function like meshgrid that would return me a grid of points linearly interpolating those four corner points? It depends on what you mean by linearly interpolating. For example, you can construct a regular grid and simply discard points outside the trapesium, or assume that the trapesium is a warped perspective on a regular grid. For the latter case, you can construct a grid like this: http://mentat.za.net/refer/np_warped_grid.png (code attached) Stéfan import numpy as np import matplotlib.pyplot as plt rx, ry = np.array([[0, 0], [1, 0], [0, 1], [1, 1]]).T tx, ty = np.array([[7, 10], [20, 10], [4, 20], [25, 20]]).T N = len(rx) U = np.zeros((2 * N, 8)) U[:N, 0] = rx U[:N, 1] = ry U[:N, 2] = 1 U[:N, 6] = -rx * tx U[:N, 7] = -ry * tx U[N:, 3] = rx U[N:, 4] = ry U[N:, 5] = 1 U[N:, 6] = -rx * ty U[N:, 7] = -ry * ty b = np.concatenate((tx, ty))[:, np.newaxis] M, res, rank, s = np.linalg.lstsq(U, b) M = np.append(M, 1).reshape((3, 3)) y, x = np.mgrid[:1:10j, :1:10j] I = np.ones(x.size) square_grid = np.column_stack((x.ravel(), y.ravel(), I)) warp_grid = M.dot(square_grid.T).T warp_grid /= warp_grid[:, 2][:, None] plt.scatter(tx, ty, c='b', s=50) plt.scatter(warp_grid[:, 0], warp_grid[:, 1], c='r', marker='x') plt.show() ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [ANN] new solver for multiobjective optimization problems
On 10 February 2012 17:59, Neal Becker ndbeck...@gmail.com wrote: And where do we find this gem? Presumably by following the hyper-links in the e-mail (non-obvious if you're using a plain-text mail client..) Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
