[Numpy-discussion] ANN: SfePy 2017.1
I am pleased to announce release 2017.1 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (limited support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -- - spline-box parametrization of an arbitrary field - conda-forge recipe (thanks to Daniel Wheeler) - fixes for Python 3.6 For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Cheers, Robert Cimrman --- Contributors to this release in alphabetical order: Siwei Chen Robert Cimrman Jan Heczko Vladimir Lukes Matyas Novak ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2016.4
I am pleased to announce release 2016.4 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (limited support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -- - support tensor product element meshes with one-level hanging nodes - improve homogenization support for large deformations - parallel calculation of homogenized coefficients and related sub-problems - evaluation of second derivatives of Lagrange basis functions For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Cheers, Robert Cimrman --- Contributors to this release in alphabetical order: Robert Cimrman Vladimir Lukes Matyas Novak ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2016.3
I am pleased to announce release 2016.3 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (limited support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker: http://github.com/sfepy/sfepy Highlights of this release -- - Python 3 support - testing with Travis CI - new classes for homogenized coefficients - using argparse instead of optparse For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Cheers, Robert Cimrman --- Contributors to this release in alphabetical order: Robert Cimrman Jan Heczko Thomas Kluyver Vladimir Lukes ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2016.2
I am pleased to announce release 2016.2 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - partial shell10x element implementation - parallel computation of homogenized coefficients - clean up of elastic terms - read support for msh file mesh format of gmsh For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman on behalf of the SfePy development team --- Contributors to this release in alphabetical order: Robert Cimrman Vladimir Lukes ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2016.1
I am pleased to announce release 2016.1 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - major simplification of finite element field code - automatic checking of shapes of term arguments - improved mesh parametrization code and documentation - support for fieldsplit preconditioners of PETSc For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman on behalf of the SfePy development team --- Contributors to this release in alphabetical order: Robert Cimrman Vladimir Lukes ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2015.4
I am pleased to announce release 2015.4 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - basic support for restart files - new type of linear combination boundary conditions - balloon inflation example For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman on behalf of the SfePy development team --- Contributors to this release in alphabetical order: Robert Cimrman Grant Stephens ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2015.3
I am pleased to announce release 2015.3 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - preliminary support for parallel computing - unified evaluation of basis functions (= isogeometric analysis fields can be evaluated in arbitrary points) - (mostly) fixed finding of reference element coordinates of physical points - several new or improved examples For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman on behalf of the SfePy development team --- Contributors to this release in alphabetical order: Robert Cimrman Vladimir Lukes ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2015.2
I am pleased to announce release 2015.2 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - major code simplification (removed element groups) - time stepping solvers updated for interactive use - improved finding of reference element coordinates of physical points - reorganized examples - reorganized installation on POSIX systems (sfepy-run script) For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Lubos Kejzlar, Vladimir Lukes, Anton Gladky, Matyas Novak ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2015.1
I am pleased to announce release 2015.1 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - support for multiple fields in isogeometric analysis - redesigned handling of solver parameters - new modal analysis example For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Lubos Kejzlar, Vladimir Lukes ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2014.4
I am pleased to announce release 2014.4 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - preliminary support for 1D problems - data probes using pyVTK library For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Lubos Kejzlar, Vladimir Lukes ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2014.3
I am pleased to announce release 2014.3 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - isogeometric analysis (IGA) speed-up by C implementation of NURBS basis evaluation - generalized linear combination boundary conditions that work between different fields/variables and support non-homogeneous periodic conditions - non-constant essential boundary conditions given by a function in IGA - reorganized and improved documentation For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Vladimir Lukes, Matyas Novak, Zhihua Ouyang, Jaroslav Vondrejc ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2014.2
I am pleased to announce release 2014.2 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method. The code is based on NumPy and SciPy packages. It is distributed under the new BSD license. This release brings a preliminary support for isogeometric analysis - a recently developed computational approach that allows using the NURBS-based domain description from CAD design tools also for approximation purposes similar to the finite element method. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - preliminary support for isogeometric analysis - improved post-processing and visualization script for time-dependent problems with adaptive time steps - three new terms For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Vladimír Lukeš ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2014.1
I am pleased to announce release 2014.1 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method. The code is based on NumPy and SciPy packages. It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - sfepy.fem was split to separate FEM-specific and general modules - lower memory usage by creating active DOF connectivities directly from field connectivities - new handling of field and variable shapes - clean up: many obsolete modules were removed, all module names follow naming conventions For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Vladimír Lukeš, Matyáš Novák, Jaroslav Vondřejc ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2013.4
I am pleased to announce release 2013.4 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method. The code is based on NumPy and SciPy packages. It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -- - simplified quadrature definition - equation sequence solver - initial support for 'plate' integration/connectivity type - script for visualization of quadrature points and weights For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Vladimír Lukeš, Jaroslav Vondřejc ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 2013.3
I am pleased to announce release 2013.3 of SfePy. Description --- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method. The code is based on NumPy and SciPy packages. It is distributed under the new BSD license. Home page: http://sfepy.org Downloads, mailing list, wiki: http://code.google.com/p/sfepy/ Git (source) repository, issue tracker: http://github.com/sfepy Highlights of this release -- - implementation of Mesh topology data structures in C - implementation of regions based on C Mesh (*) - MultiProblem solver for conjugate solution of subproblems - new advanced examples (vibro-acoustics, Stokes flow with slip conditions) (*) Warning: region selection syntax has been changed in a principal way, see [1]. Besides the simple renaming, all regions meant for boundary conditions or boundary/surface integrals need to have their kind set explicitly to 'facet' (or 'edge' in 2D, 'face' in 3D). [1] http://sfepy.org/doc-devel/users_guide.html#regions For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Vladimír Lukeš ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: SfePy 2013.3
Dear Josè, On 09/18/2013 07:10 PM, Josè Luis Mietta wrote: Dear Robert. Im intresting in modeling mechanical deformation of magnetorheological elastomers (material formed by inorganic chains inserting in a polymeric matrix -see figure 2 in the attached file-). The inorganic chais are like pearl necklace (are formed by saligned pheres ). Can I use this tool (SfePy) for modeling the mechanical deformation (i.e.: shape deformation by appling a force in longitudinal and transversal direction respect to the inorganic chains)? I guess so :) You are interested in modeling a single chain, not the whole microstructure, right? (BTW. if you are interested in modeling the whole box with many chains, check [1]). How can I do that? These are general steps, not particular to sfepy: First you need to create a FE mesh out of the CT data. There are probably many packages that can do that. I am familiar (and know it works) with [2], which has been created by Vladimir Lukes, another sfepy developer. Then you need to define the problem - specify boundary conditions, choose FE approximation, apply loads and finally solve and visualize. Im a newby user of python and I dont know anything about simple finite elements analysis. What documentation (books or other) do yo recommend for my introduction in this topic? Check the wiki page [3] and possibly the references given there. If you have other questions, we can discuss off-list, or on the sfepy mailing list [4]. Best regards, r. [1] http://parfe.sourceforge.net/ [2] http://sfepy.org/dicom2fem/ [3] http://en.wikipedia.org/wiki/Finite_element_method [4] https://groups.google.com/forum/#!forum/sfepy-devel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Incrementing with advanced indexing: why don't repeated indexes repeatedly increment?
On 06/06/2012 05:06 PM, Nathaniel Smith wrote: On Wed, Jun 6, 2012 at 9:48 AM, John Salvatier jsalv...@u.washington.edu wrote: Hello, I've noticed that If you try to increment elements of an array with advanced indexing, repeated indexes don't get repeatedly incremented. For example: In [30]: x = zeros(5) In [31]: idx = array([1,1,1,3,4]) In [32]: x[idx] += [2,4,8,10,30] In [33]: x Out[33]: array([ 0., 8., 0., 10., 30.]) I would intuitively expect the output to be array([0,14, 0,10,30]) since index 1 is incremented by 2+4+8=14, but instead it seems to only increment by 8. What is numpy actually doing here? The authors of Theano noticed this behavior a while ago so they python loop through the values in idx (this kind of calculation is necessary for calculating gradients), but this is a bit slow for my purposes, so I'd like to figure out how to get the behavior I expected, but faster. I'm also not sure how to navigate the numpy codebase, where would I look for the code responsible for this behavior? Strictly speaking, it isn't actually in the numpy codebase at all -- what's happening is that the Python interpreter sees this code: x[idx] += vals and then it translates it into this code before running it: tmp = x.__getitem__(idx) tmp = tmp.__iadd__(vals) x.__setitem__(idx, tmp) So you can find the implementations of the ndarray methods __getitem__, __iadd__, __setitem__ (they're called array_subscript_nice, array_inplace_add, and array_ass_sub in the C code), but there's no way to fix them so that this works the way you want it to, because there's no way for __iadd__ to know that the temporary values that it's working with are really duplicate copies of the same value in the original array. It would be nice if numpy had some sort of standard API for doing what you want, but not sure what a good API would look like, and someone would have to implement it. This operation is also heavily used for the finite element assembling, and a similar question has been raised already several times (e.g. http://old.nabble.com/How-to-assemble-large-sparse-matrices-effectively-td33833855.html). So why not adding a function np.assemble()? r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Incrementing with advanced indexing: why don't repeated indexes repeatedly increment?
On 06/06/2012 05:34 PM, Nathaniel Smith wrote: On Wed, Jun 6, 2012 at 4:30 PM, Robert Cimrmancimrm...@ntc.zcu.cz wrote: On 06/06/2012 05:06 PM, Nathaniel Smith wrote: On Wed, Jun 6, 2012 at 9:48 AM, John Salvatier jsalv...@u.washington.eduwrote: Hello, I've noticed that If you try to increment elements of an array with advanced indexing, repeated indexes don't get repeatedly incremented. For example: In [30]: x = zeros(5) In [31]: idx = array([1,1,1,3,4]) In [32]: x[idx] += [2,4,8,10,30] In [33]: x Out[33]: array([ 0., 8., 0., 10., 30.]) I would intuitively expect the output to be array([0,14, 0,10,30]) since index 1 is incremented by 2+4+8=14, but instead it seems to only increment by 8. What is numpy actually doing here? The authors of Theano noticed this behavior a while ago so they python loop through the values in idx (this kind of calculation is necessary for calculating gradients), but this is a bit slow for my purposes, so I'd like to figure out how to get the behavior I expected, but faster. I'm also not sure how to navigate the numpy codebase, where would I look for the code responsible for this behavior? Strictly speaking, it isn't actually in the numpy codebase at all -- what's happening is that the Python interpreter sees this code: x[idx] += vals and then it translates it into this code before running it: tmp = x.__getitem__(idx) tmp = tmp.__iadd__(vals) x.__setitem__(idx, tmp) So you can find the implementations of the ndarray methods __getitem__, __iadd__, __setitem__ (they're called array_subscript_nice, array_inplace_add, and array_ass_sub in the C code), but there's no way to fix them so that this works the way you want it to, because there's no way for __iadd__ to know that the temporary values that it's working with are really duplicate copies of the same value in the original array. It would be nice if numpy had some sort of standard API for doing what you want, but not sure what a good API would look like, and someone would have to implement it. This operation is also heavily used for the finite element assembling, and a similar question has been raised already several times (e.g. http://old.nabble.com/How-to-assemble-large-sparse-matrices-effectively-td33833855.html). So why not adding a function np.assemble()? I read that message, but I don't see what it has to do with this discussion? It seemed to be about fast ways to assign dense matrices into sparse matrices, not fast ways of applying in-place arithmetic to specific spots in a dense matrix. Yes (in that thread), but it applies also adding/assembling vectors into a global vector - this is just x[idx] += vals. I linked that discussion as that was recent enough for me to recall it, but there were other. Anyway, my point was that a having a function with the adding semantics in NumPy would be handy. r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Incrementing with advanced indexing: why don't repeated indexes repeatedly increment?
On 06/06/2012 06:35 PM, Robert Kern wrote: On Wed, Jun 6, 2012 at 4:52 PM, Robert Cimrmancimrm...@ntc.zcu.cz wrote: Yes (in that thread), but it applies also adding/assembling vectors into a global vector - this is just x[idx] += vals. I linked that discussion as that was recent enough for me to recall it, but there were other. Anyway, my point was that a having a function with the adding semantics in NumPy would be handy. x += numpy.bincount(idx, vals, minlength=len(x)) Nice! Looking at the C source, it seems it should be pretty efficient for this task. r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] The NumPy Mandelbrot code 16x slower than Fortran
On 01/23/12 13:51, Sturla Molden wrote:
Den 23.01.2012 13:09, skrev Sebastian Haase:
I would think that interactive zooming would be quite nice
(illuminating) and for that 13 secs would not be tolerable
Well... it's not at the top of my priority list ... ;-)
Sure, that comes under the 'fast enough' issue. But even Fortran might
be too slow here?
For zooming Mandelbrot I'd use PyOpenGL and a GLSL fragment shader
(which would be a text string in Python):
madelbrot_fragment_shader =
uniform sampler1D tex;
uniform vec2 center;
uniform float scale;
uniform int iter;
void main() {
vec2 z, c;
c.x = 1. * (gl_TexCoord[0].x - 0.5) * scale - center.x;
c.y = (gl_TexCoord[0].y - 0.5) * scale - center.y;
int i;
z = c;
for(i=0; iiter; i++) {
float x = (z.x * z.x - z.y * z.y) + c.x;
float y = (z.y * z.x + z.x * z.y) + c.y;
if((x * x + y * y) 4.0) break;
z.x = x;
z.y = y;
}
gl_FragColor = texture1D(tex, (i == iter ? 0.0 : float(i)) / 100.0);
}
The rest is just boiler-plate OpenGL...
Sources:
http://nuclear.mutantstargoat.com/articles/sdr_fract/
http://pyopengl.sourceforge.net/context/tutorials/shader_1.xhtml
Off-topic comment: Or use some algorithmic cleverness, see [1]. I recall Xaos
had interactive, extremely fast a fluid fractal zooming more than 10 (or 15?)
years ago (- on a laughable hardware by today's standards).
r.
[1] http://wmi.math.u-szeged.hu/xaos/doku.php
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy.distutils quirk
On 10/18/11 22:13, Ralf Gommers wrote: On Tue, Oct 11, 2011 at 4:59 PM, Robert Cimrmancimrm...@ntc.zcu.cz wrote: Hi, I have now spent several hours hunting down a major slowdown of my code caused (apparently) by using config.add_library() for a reusable part of C source files instead of just config.add_extension(). The reason of the slowdown was different, but hard to discern, naming of options and silent ignoring of non-existing ones: add_library() : extra_compiler_args add_extension() : extra_compile_args Other build keys used for the same purpose also differ. A bug to be reported, or is this going to be solved by going bento? Bento will use saner names I hope, but filing a bug can't hurt. We're still fixing distutils issues. Ok. I am getting internal server error at http://projects.scipy.org/numpy/newticket now - I will wait some time and try again. r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Cellular Automata Neighborhoods Numpy
Hi Michael, You can find the full game of life script at [1]. There is also Belousov-Zhabotinsky cellular automaton. Both have a strided version. r. [1] http://docs.sfepy.org/scientific-python-tutorial/examples - Reply message - From: Michael Mersky m...@mydis.org To: numpy-discussion@scipy.org Subject: [Numpy-discussion] Cellular Automata Neighborhoods amp; Numpy Date: Sun, Mar 20, 2011 03:35 Hello, I stumbled upon this group tonight (http://mail.scipy.org/pipermail/numpy-discussion/2010-October/053420.html) while searching Google for examples of Cellular Automata(CA) using Numpy. The Game of Life Strides example looks great, but I don't fully comprehend how this example is working: http://scipy.org/Cookbook/GameOfLifeStrides If I have: world=arange(9).reshape(3,3) world array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) how could I as_strided(world,shape?,strides) such that my 9 cell neighborhood at [0][0] would be something like this (i.e. wrapped): array([[8,6,7], [2,0,1], [5,3,4]]) Greatly appreciate your help! -m___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] using loadtxt() for given number of rows?
On Mon, 14 Feb 2011, Stéfan van der Walt wrote: Hi Robert On Mon, Jan 31, 2011 at 2:39 PM, Robert Cimrman cimrm...@ntc.zcu.cz wrote: It seems to me, that an additional parameter to loadtxt(), say nrows or numrows, would do the job, so that the function does not try reading the entire file. Another possibility would be to raise an exception as it is now, but also to return the data succesfully read so far. You can always read chunks of the file into StringIO objects, and pass those into loadtxt. I think your request makes sense though, given that a person can already skip lines at the top of the file. Thanks for the tip, Stéfan! I have solved my problem by using np.fromfile(), as suggested by Chris Barker, because I know the number of items to read in advance. I also think that functionality suitable for loadtxt(), so I created the NumPy ticket 1731. r.___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] using loadtxt() for given number of rows?
On Mon, 31 Jan 2011, Christopher Barker wrote: On 1/31/11 4:39 AM, Robert Cimrman wrote: I work with text files which contain several arrays separated by a few lines of other information, for example: POINTS 4 float -5.00e-01 -5.00e-01 0.00e+00 5.00e-01 -5.00e-01 0.00e+00 5.00e-01 5.00e-01 0.00e+00 -5.00e-01 5.00e-01 0.00e+00 CELLS 2 8 3 0 1 2 3 2 3 0 I have used custom Python code with loops to read similar files, so the speed was not too good. Now I wonder if it would be possible to use the numpy.loadtxt() function for the array-like parts. It supports passing an open file object in, which is good, but it wants to read the entire file, which does not work in this case. It seems to me, that an additional parameter to loadtxt(), say nrows or numrows, would do the job, I agree that that would be a useful feature. However, I'm not sure it would help performance much -- I think loadtxt is written in python as well. I see. Anyway, it would allow me to reduce my code size, which counts as well to be a good thing. So there is now a new enhancement ticket [1]. One option in the meantime. If you know how many rows, you presumable know how many items on each row. IN that case, you can use: np.fromfile(open_file, sep=' ', count=num_items_to_read) It'll only work for multi-line text if the separator is whitespace, which it was in your example. But if it does, it should be pretty fast. Good idea, the prerequisites are not met always, but often enough. Thanks! r. [1] http://projects.scipy.org/numpy/ticket/1731 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] using loadtxt() for given number of rows?
Hi, I work with text files which contain several arrays separated by a few lines of other information, for example: POINTS 4 float -5.00e-01 -5.00e-01 0.00e+00 5.00e-01 -5.00e-01 0.00e+00 5.00e-01 5.00e-01 0.00e+00 -5.00e-01 5.00e-01 0.00e+00 CELLS 2 8 3 0 1 2 3 2 3 0 (yes, that's the legacy VTK format, but take it just as an example) I have used custom Python code with loops to read similar files, so the speed was not too good. Now I wonder if it would be possible to use the numpy.loadtxt() function for the array-like parts. It supports passing an open file object in, which is good, but it wants to read the entire file, which does not work in this case. It seems to me, that an additional parameter to loadtxt(), say nrows or numrows, would do the job, so that the function does not try reading the entire file. Another possibility would be to raise an exception as it is now, but also to return the data succesfully read so far. What do you think? Is this worth a ticket? r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Commit rights on github
Hi Fernando, On Tue, 12 Oct 2010, Fernando Perez wrote: 2010/10/12 Stéfan van der Walt ste...@sun.ac.za: - Then: merge into master, getting a fast-forward merge if possible - Push back to github When I have large changes that consist of several commits on a single topic, I normally explicitly ask for a non-fast-forward merge, so that the history is not contaminated by a whole slew of commit messages. My rule of thumb is: one or two commits, fast-forward is OK, three or more, I merge explicitly with --no-ff to make it easier to see them together with their own little branch. In case anyone is interested, we're having precisely this discussion on ipython-dev (where we've been using DVCS for over 2 years and have had ample opportunity to make every mistake under the sun, but now we're quite happy with our workflow and just fine-tuning it): http://mail.scipy.org/pipermail/ipython-dev/2010-October/006746.html Very interesting reading! I have always used tags to mark branches that were fast-forward-merged to master, but --no-ff makes it easier to see the actual branch. Looking forward to the discussion being included in your dev docs, r.___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] allclose() does not check shape of inputs
Hi, I think this is a bug: In [16]: np.allclose([1.0, 1.0], [1.1], rtol=0.1, atol=0.0) Out[16]: True Shall I create a ticket? r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] allclose() does not check shape of inputs
Pauli Virtanen wrote: Fri, 13 Nov 2009 11:54:51 +0100, Robert Cimrman wrote: I think this is a bug: In [16]: np.allclose([1.0, 1.0], [1.1], rtol=0.1, atol=0.0) Out[16]: True It's broadcasting. I'm not sure it is a bug: np.allclose([1.0, 1.0], [1.1, 1.1, 1.1], rtol=0.1, atol=0.0) False I see, I hit the corner case... Sorry for the noise! r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] numpy distutils and cython
Hi, I am using numpy distutils to build the extension modules of a project, which have been so far written in C, and wrapped by SWIG. Now I would like to try cython (as everynone!), but still be able to use the numpy distutils. I have found the thread [1], which offers some solution, but it does not mention using the Configuration class from numpy.distutils.misc_util which I use. How can I tell Configuration to use Cython instead of Pyrex? thanks! r. [1] http://old.nabble.com/problem-with-numpy.distutils-and-Cython-td25100957.html ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] intersect1d for N input arrays
Hi Martin, thanks for your ideas and contribution. A few notes: I would let intersect1d as it is, and created a new function with another name for that (any proposals?). Considering that most of arraysetops functions are based on sort, and in particular here that an intersection array is (usually) smaller than each of the input arrays, it might be better just to call intersect1d repeatedly for each array and the result of the previous call, accumulating the intersection. r. Martin Spacek wrote: I have a list of many arrays (in my case each is unique, ie has no repeated elements), and I'd like to extract the intersection of all of them, all in one go. I'm running numpy 1.3.0, but looking at today's rev of numpy.lib.arraysetops (http://svn.scipy.org/svn/numpy/trunk/numpy/lib/arraysetops.py), I see intersect1d has changed. Just a note: the example used in the docstring implies that the two arrays need to be the same length, which isn't the case. Maybe it would be good to change the example to two arrays of different lengths. intersect1d takes exactly 2 arrays. I've modified it a little to take the intersection of any number of 1D arrays (of any length), in a list or tuple. It seems to work fine, but could use more testing. Here it is with most of the docs stripped. Feel free to use it, although I suppose for symmetry, many of the other functions in arraysetops.py would also have to be modified to work with N arrays: def intersect1d(arrays, assume_unique=False): Find the intersection of any number of 1D arrays. Return the sorted, unique values that are in all of the input arrays. Adapted from numpy.lib.arraysetops.intersect1d N = len(arrays) arrays = list(arrays) # allow assignment if not assume_unique: for i, arr in enumerate(arrays): arrays[i] = np.unique(arr) aux = np.concatenate(arrays) # one long 1D array aux.sort() # sorted shift = N-1 return aux[aux[shift:] == aux[:-shift]] ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] simple indexing question
Neal Becker wrote: I have an array: In [12]: a Out[12]: array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) And a selection array: In [13]: b Out[13]: array([1, 1, 1, 1, 1]) I want a 1-dimensional output, where the array b selects an element from each column of a, where if b[i]=0 select element from 0th row of a and if b[i]=k select element from kth row of a. Easy way to do this? (Not a[b], that gives 5x5 array output) It might be stupid, but it works... In [51]: a Out[51]: array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) In [52]: b = [0,1,0,1,0] In [53]: a.T.flat[a.shape[0]*np.arange(a.shape[1])+b] Out[53]: array([0, 6, 2, 8, 4]) cheers, r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] simple indexing question
josef.p...@gmail.com wrote: On Wed, Sep 23, 2009 at 11:12 AM, Neal Becker ndbeck...@gmail.com wrote: Robert Cimrman wrote: Neal Becker wrote: I have an array: In [12]: a Out[12]: array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) And a selection array: In [13]: b Out[13]: array([1, 1, 1, 1, 1]) I want a 1-dimensional output, where the array b selects an element from each column of a, where if b[i]=0 select element from 0th row of a and if b[i]=k select element from kth row of a. Easy way to do this? (Not a[b], that gives 5x5 array output) It might be stupid, but it works... In [51]: a Out[51]: array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) In [52]: b = [0,1,0,1,0] In [53]: a.T.flat[a.shape[0]*np.arange(a.shape[1])+b] Out[53]: array([0, 6, 2, 8, 4]) cheers, r. Thanks. Is there really no more elegant solution? How about this? a array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) b array([0, 1, 0, 1, 0]) a[b,np.arange(a.shape[1])] array([0, 6, 2, 8, 4]) So it was stupid :) well, time to go home, r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] improving arraysetops
Hi Neil, Neil Crighton wrote: What about merging unique and unique1d? They're essentially identical for an array input, but unique uses the builtin set() for non-array inputs and so is around 2x faster in this case - see below. Is it worth accepting a speed regression for unique to get rid of the function duplication? (Or can they be combined?) unique1d can return the indices - can this be achieved by using set(), too? No, set() can't return the indices as far as I know. The implementation for arrays is the same already, IMHO, so I would prefer adding return_index, return_inverse to unique (automatically converting input to array, if necessary), and deprecate unique1d. We can view it also as adding the set() approach to unique1d, when the return_index, return_inverse arguments are not set, and renaming unique1d - unique. This sounds good. If you don't have time to do it, I don't mind having a go at writing a patch to implement these changes (deprecate the existing unique1d, rename unique1d to unique and add the set approach from the old unique, and the other changes mentioned in http://projects.scipy.org/numpy/ticket/1133). That would be really great - I will not be online starting tomorrow till the end of next week (more or less), so I can really look at the issue after I return. [...] UnicodeEncodeError: 'ascii' codec can't encode character u'\xb5' in position 28: ordinal not in range(128) It disappears after increasing the array size, or the integer size. In [39]: np.__version__ Out[39]: '1.4.0.dev7047' r. Weird! From the error message, it looks like a problem with ipython's timeit function rather than unique. I can't reproduce it on my machine (numpy 1.4.0.dev, r7059; IPython 0.10.bzr.r1163 ). True, I have ipython 0.9.1, that might cause the problem. cheers, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] improving arraysetops
Neil Crighton wrote: Robert Cimrman cimrman3 at ntc.zcu.cz writes: Hi, I am starting a new thread, so that it reaches the interested people. Let us discuss improvements to arraysetops (array set operations) at [1] (allowing non-unique arrays as function arguments, better naming conventions and documentation). r. [1] http://projects.scipy.org/numpy/ticket/1133 Hi, These changes looks good to me. For point (1) I think we should fold the unique and _nu code into a single function. For point (3) I like in1d - it's shorter than isin1d but is still clear. yes, the _nu functions will be useless then, their bodies can be moved into the generic functions. What about merging unique and unique1d? They're essentially identical for an array input, but unique uses the builtin set() for non-array inputs and so is around 2x faster in this case - see below. Is it worth accepting a speed regression for unique to get rid of the function duplication? (Or can they be combined?) unique1d can return the indices - can this be achieved by using set(), too? The implementation for arrays is the same already, IMHO, so I would prefer adding return_index, return_inverse to unique (automatically converting input to array, if necessary), and deprecate unique1d. We can view it also as adding the set() approach to unique1d, when the return_index, return_inverse arguments are not set, and renaming unique1d - unique. Neil In [24]: l = list(np.random.randint(100, size=1)) In [25]: %timeit np.unique1d(l) 1000 loops, best of 3: 1.9 ms per loop In [26]: %timeit np.unique(l) 1000 loops, best of 3: 793 µs per loop In [27]: l = list(np.random.randint(100, size=100)) In [28]: %timeit np.unique(l) 10 loops, best of 3: 78 ms per loop In [29]: %timeit np.unique1d(l) 10 loops, best of 3: 233 ms per loop I have found a strange bug in unique(): In [24]: l = list(np.random.randint(100, size=1000)) In [25]: %timeit np.unique(l) --- UnicodeEncodeErrorTraceback (most recent call last) /usr/lib64/python2.5/site-packages/IPython/iplib.py in ipmagic(self, arg_s) 951 else: 952 magic_args = self.var_expand(magic_args,1) -- 953 return fn(magic_args) 954 955 def ipalias(self,arg_s): /usr/lib64/python2.5/site-packages/IPython/Magic.py in magic_timeit(self, parameter_s) 1829 precision, 1830 best * scaling[order], - 1831 units[order]) 1832 if tc tc_min: 1833 print Compiler time: %.2f s % tc UnicodeEncodeError: 'ascii' codec can't encode character u'\xb5' in position 28: ordinal not in range(128) It disappears after increasing the array size, or the integer size. In [39]: np.__version__ Out[39]: '1.4.0.dev7047' r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] setmember1d_nu
Neil Crighton wrote: Robert Cimrman cimrman3 at ntc.zcu.cz writes: I'd really like to see the setmember1d_nu function in ticket 1036 get into numpy. There's a patch waiting for review that including tests: http://projects.scipy.org/numpy/ticket/1036 Is there anything I can do to help get it applied? I guess I could commit it, if you review the patch and it works for you. Obviously, I cannot review it myself, but my SVN access may still work :) Thanks for the review, it is in! r. Great - thanks! People often post to the list asking for this functionality, so it's nice to get it into numpy (whatever it ends up being called). Thank you for starting the discussion :) ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] improving arraysetops
Hi, I am starting a new thread, so that it reaches the interested people. Let us discuss improvements to arraysetops (array set operations) at [1] (allowing non-unique arrays as function arguments, better naming conventions and documentation). r. [1] http://projects.scipy.org/numpy/ticket/1133 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] extract elements of an array that are contained in another array?
Hi Josef, thanks for the summary! I am responding below, later I will make an enhancement ticket. josef.p...@gmail.com wrote: On Sat, Jun 6, 2009 at 4:42 AM, Neil Crighton neilcrigh...@gmail.com wrote: Robert Cimrman cimrman3 at ntc.zcu.cz writes: Anne Archibald wrote: 1. add a keyword argument to intersect1d assume_unique; if it is not present, check for uniqueness and emit a warning if not unique 2. change the warning to an exception Optionally: 3. change the meaning of the function to that of intersect1d_nu if the keyword argument is not present 1. merge _nu version into one function --- You mean something like: def intersect1d(ar1, ar2, assume_unique=False): if not assume_unique: return intersect1d_nu(ar1, ar2) else: ... # the current code intersect1d_nu could be still exported to numpy namespace, or not. +1 - from the user's point of view there should just be intersect1d and setmember1d (i.e. no '_nu' versions). The assume_unique keyword Robert suggests can be used if speed is a problem. + 1 on rolling the _nu versions this way into the plain version, this would avoid a lot of the confusion. It would not be a code breaking API change for existing correct usage (but some speed regression without adding keyword) +1 depreciate intersect1d_nu ^^ intersect1d_nu could be still exported to numpy namespace, or not. I would say not, if they are the default branch of the non _nu version +1 on depreciation +0 2. alias as in - I really like in1d (no underscore) as a new name for setmember1d_nu. inarray is another possibility. I don't like 'ain'; 'a' in front of 'in' detracts from readability, unlike the extra a in arange. I don't like the extra as either, ones name spaces are commonly used alias setmember1d_nu as `in1d` or `isin1d`, because the function is a in and not a set operation +1 +1 3. behavior of other set functions --- guarantee that setdiff1d works for non-unique arrays (even when implementation changes), and change documentation +1 +1, it is useful for non-unique arrays. need to check other functions ^^ union1d: works for non-unique arrays, obvious from source Yes. setxor1d: requires unique arrays np.setxor1d([1,2,3,3,4,5], [0,0,1,2,2,6]) array([2, 4, 5, 6]) np.setxor1d(np.unique([1,2,3,3,4,5]), np.unique([0,0,1,2,2,6])) array([0, 3, 4, 5, 6]) setxor: add keyword option and call unique by default +1 for symmetry +1 - you mean np.setxor1d(np.unique(a), np.unique(b)) to become np.setxor1d(a, b, assume_unique=False), right? ediff1d and unique1d are defined for non-unique arrays yes 4. name of keyword intersect1d(ar1, ar2, assume_unique=False) alternative isunique=False or just unique=False +1 less to write We should look at other functions in numpy (and/or scipy), what is a common scheme here. -1e-1 to the proposed names, as isunique is singular only, and unique=False does not show clearly the intent for me. What about ar1_unique=False, ar2_unique=False - to address each argument specifically? 5. module name --- rename arraysetops to something easier to read like setfun. I think it would only affect internal changes since all functions are exported to the main numpy name space +1e-4 (I got used to arrayse_tops) +0 (internal change only). Other numpy/scipy submodules containing a bunch of functions are called *pack (fftpack, arpack, lapack), *alg (linalg), *utils. *fun is used comonly in the matlab world. 5. keep docs in sync with correct usage - obvious +1 thanks, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] extract elements of an array that are contained in another array?
Robert Cimrman wrote: Hi Josef, thanks for the summary! I am responding below, later I will make an enhancement ticket. Done, see http://projects.scipy.org/numpy/ticket/1133 r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] setmember1d_nu
Robert Cimrman wrote: Hi Neil, Neil Crighton wrote: Hi all, I posted this message couple of days ago, but gmane grouped it with an old thread and it hasn't shown up on the front page. So here it is again... I'd really like to see the setmember1d_nu function in ticket 1036 get into numpy. There's a patch waiting for review that including tests: http://projects.scipy.org/numpy/ticket/1036 Is there anything I can do to help get it applied? I guess I could commit it, if you review the patch and it works for you. Obviously, I cannot review it myself, but my SVN access may still work :) Thanks for the review, it is in! r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] extract elements of an array that are contained in another array?
josef.p...@gmail.com wrote: On Fri, Jun 5, 2009 at 1:48 AM, Robert Cimrman cimrm...@ntc.zcu.cz wrote: josef.p...@gmail.com wrote: On Thu, Jun 4, 2009 at 4:30 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 04, 2009 at 10:27:11PM +0200, Kim Hansen wrote: in(b) or in_iterable(b) method, such that you could do a.in(b) which would return a boolean array of the same shape as a with elements true if the equivalent a members were members in the iterable b. That would really by what I would be looking for. Just using in might promise more than it does, eg. it works only for one dimensional arrays, maybe in1d. With in, I would expect a generic function as in python that works with many array types and dimensions. (But I haven't checked whether it would work with a 1d structured array or object array.) I found arraysetops because of unique1d, but I didn't figure out what the subpackage really does, because I was reading arrayse-tops instead of array-set-ops I am bad in choosing names, but note that numpy sub-modules usually do not use underscores, so array_set_ops would not fit well. I would have chosen something like setfun. Since this is in numpy that sets refers to arrays should be implied. Yes, good idea. I am not sure how to proceed, if people agree (name contest is open!) What about making an alias name setfun, and deprecate the name arraysetops? BTW, for the docs, I haven't found a counter example where np.setdiff1d gives the wrong answer for non-unique arrays. In [4]: np.setmember1d( [1, 1, 2, 4, 2], [3, 2, 4] ) Out[4]: array([ True, False, True, True, True], dtype=bool) setdiff1ddiff not member Looking at the source, I think setdiff always works even if for non-unique arrays. Whoops, sorry. setdiff1d seems really to work for non-unique arrays - it relies on the behaviour above though :) - there is always one correct False even for repeated entries in the first array. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] setmember1d_nu
Hi Neil, Neil Crighton wrote: Hi all, I posted this message couple of days ago, but gmane grouped it with an old thread and it hasn't shown up on the front page. So here it is again... I'd really like to see the setmember1d_nu function in ticket 1036 get into numpy. There's a patch waiting for review that including tests: http://projects.scipy.org/numpy/ticket/1036 Is there anything I can do to help get it applied? I guess I could commit it, if you review the patch and it works for you. Obviously, I cannot review it myself, but my SVN access may still work :) r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] extract elements of an array that are contained in another array?
Alan G Isaac wrote: On 6/4/2009 10:50 AM josef.p...@gmail.com apparently wrote: intersect1d gives set intersection if both arrays have only unique elements (i.e. are sets). I thought the naming is pretty clear: intersect1d(a,b) set intersection if a and b with unique elements intersect1d_nu(a,b) set intersection if a and b with non-unique elements setmember1d(a,b) boolean index array for a of set intersection if a and b with unique elements setmember1d_nu(a,b) boolean index array for a of set intersection if a and b with non-unique elements a array([1, 1, 2, 3, 3, 4]) b array([1, 4, 4, 4]) np.intersect1d_nu(a,b) array([1, 4]) That is, intersect1d_nu is the actual set intersection function. (I.e., intersect1d and intersect1d_nu would most naturally have swapped names.) That is why the appended _nu will not communicate what was intended. (I.e., setmember1d_nu will not be a match for intersect1d_nu.) The naming should express this: intersect1d expects its arguments are sets, intersect1d_nu does not. A set has unique elements by definition. cheers, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] extract elements of an array that are contained in another array?
josef.p...@gmail.com wrote: On Thu, Jun 4, 2009 at 2:58 PM, Alan G Isaac ais...@american.edu wrote: On 6/4/2009 1:27 PM josef.p...@gmail.com apparently wrote: Note: there are two versions of the docs for np.intersect1d, the currently published docs which describe the actual behavior (for the non-unique case), and the new docs on the doc editor http://docs.scipy.org/numpy/docs/numpy.lib.arraysetops.intersect1d/ that describe the intended usage of the functions, which also corresponds closer to the original source docstring (http://docs.scipy.org/numpy/docs/numpy.lib.arraysetops.intersect1d/?revision=-227 ). that's my interpretation Again, the distributed docs do *not* describe the actual behavior for the non-unique case. E.g., np.intersect1d([1,1,2,3,3,4], [1,4]) array([1, 1, 3, 4]) Might this is a better example of failure than the one in the doc editor? Thanks, that's a very clear example of a wrong answer, and it removes the question whether the function makes any sense for the non-unique case. I changed the example in the doc editor to this one. It will hopefully merged with the source at the next update. Thank you Josef! r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] extract elements of an array that are contained in another array?
Kim Hansen wrote: Concerning the name setmember1d_nu, I personally find it quite verbose and not the name I would expect as a non-insider coming to numpy and not knowing all the names of the more special hidden-away functions and not being a python-wiz either. To explain the naming: those names are used in matlab for functions of similar functionality. If better names are found, I am not against. What I particularly do not like is the _nu suffix (yes, blame me). r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] extract elements of an array that are contained in another array?
Anne Archibald wrote: 2009/6/4 josef.p...@gmail.com: intersect1d should throw a domain error if you give it arrays with non-unique elements, which is not done for speed reasons It seems to me that this is the basic source of the problem. Perhaps this can be addressed? I realize maintaining compatibility with the current behaviour is necessary, so how about a multistage deprecation: 1. add a keyword argument to intersect1d assume_unique; if it is not present, check for uniqueness and emit a warning if not unique 2. change the warning to an exception Optionally: 3. change the meaning of the function to that of intersect1d_nu if the keyword argument is not present One could do something similar with setmember1d. This would remove the pitfall of the 1d assumption and the wart of the _nu names without hampering performance for people who know they have unique arrays and are in a hurry. You mean something like: def intersect1d(ar1, ar2, assume_unique=False): if not assume_unique: return intersect1d_nu(ar1, ar2) else: ... # the current code intersect1d_nu could be still exported to numpy namespace, or not. I like this. I do not undestand, however, what you mean by remove the pitfall of the 1d assumption? cheers, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] extract elements of an array that are contained in another array?
josef.p...@gmail.com wrote: On Thu, Jun 4, 2009 at 4:30 PM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, Jun 04, 2009 at 10:27:11PM +0200, Kim Hansen wrote: in(b) or in_iterable(b) method, such that you could do a.in(b) which would return a boolean array of the same shape as a with elements true if the equivalent a members were members in the iterable b. That would really by what I would be looking for. Just using in might promise more than it does, eg. it works only for one dimensional arrays, maybe in1d. With in, I would expect a generic function as in python that works with many array types and dimensions. (But I haven't checked whether it would work with a 1d structured array or object array.) I found arraysetops because of unique1d, but I didn't figure out what the subpackage really does, because I was reading arrayse-tops instead of array-set-ops I am bad in choosing names, but note that numpy sub-modules usually do not use underscores, so array_set_ops would not fit well. BTW, for the docs, I haven't found a counter example where np.setdiff1d gives the wrong answer for non-unique arrays. In [4]: np.setmember1d( [1, 1, 2, 4, 2], [3, 2, 4] ) Out[4]: array([ True, False, True, True, True], dtype=bool) r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] building inplace with numpy.distutils?
Hi (David)! I am evaluating numpy.distutils as a build/install system for my project - is it possible to build the extension modules in-place so that the project can be used without installing it? A pointer to documentation concerning this would be handy... Currently I use a regular Makefile for the build, which works quite well, but is not very portable and does not solve the package installation. Otherwise let me say that numpy.distutils work very well, much better than the plain old distutils. Best regards, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building inplace with numpy.distutils?
David Cournapeau wrote: Robert Cimrman wrote: Hi (David)! I am evaluating numpy.distutils as a build/install system for my project - is it possible to build the extension modules in-place so that the project can be used without installing it? A pointer to documentation concerning this would be handy... Currently I use a regular Makefile for the build, which works quite well, but is not very portable and does not solve the package installation. Otherwise let me say that numpy.distutils work very well, much better than the plain old distutils. In-place builds can be setup with the -i option: python setup.py build_ext -i I think it is a plain distutils option. I have tried python setup.py build --inplace which did not work, and --help helped neither, that is why I asked here. But I was close :) thank you! r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fwd: Fast numpy array visualization
Stéfan van der Walt wrote: Hi all, Nicolas Rougier is doing some fun things with Pyglet and IPython! Awesome! r. -- Forwarded message -- From: Nicolas Rougier Date: 2009/4/3 Subject: Fast numpy array visualization To: pyglet-users Hi all, I've adapted the code from the ipython/python terminal to display 2d numpy arrays (using texture) directly within the terminal. Sources screenshots: http://www.loria.fr/~rougier/glnumpy.html Nicolas ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [Announce] Numpy 1.3.0 rc1
Hi, It might be too late (I was off-line last week), but anyway: I have set the milestone for the ticket 1036 [1] to 1.4, but it does not change the existing functionality, brings some new one, and the tests pass, so I wonder if it could get it into the 1.3 release? cheers, r. [1] http://projects.scipy.org/numpy/ticket/1036 David Cournapeau wrote: Hi, I am pleased to announce the release of the rc1 for numpy 1.3.0. You can find source tarballs and installers for both Mac OS X and Windows on the sourceforge page: https://sourceforge.net/projects/numpy/ https://sourceforge.net/projects/numpy/ The release note for the 1.3.0 release are below, The Numpy developers ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] setmember1d_nu
Re-hi! Robert Cimrman wrote: Hi all, I have added to the ticket [1] a script that compares the proposed setmember1d_nu() implementations of Neil and Kim. Comments are welcome! [1] http://projects.scipy.org/numpy/ticket/1036 I have attached a patch incorporating the solution that the involved people agreed on, so review, please. best regards, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] setmember1d_nu
Hi all, I have added to the ticket [1] a script that compares the proposed setmember1d_nu() implementations of Neil and Kim. Comments are welcome! [1] http://projects.scipy.org/numpy/ticket/1036 r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy array in iterable
Kim Hansen wrote: Hi again It turned out not to be quite good enough as is, as it requires unique values for both arrays. Whereas this is often true for the second argument, it is never true for the first argument in my use case, and I struggled with that for some time until i realized I could use unique1d with the rever_index optional parameter set True def ismember(totest, members) A setmember1d, which works for totest arrays with duplicate values uniques_in_test, rev_idx = unique1d(totest, return_inverse=True) uniques_in_members_mask = setmember1d(uniques_in_test, members) # Use this instead is members is not unique # uniques_in_members_mask = setmember1d(uniques_in_test, unique1d(members)) return uniques_in_members_mask[rev_idx] I saw someone else providing an alternative implementation of this, which was longer and included a loop. I do not know which is the most efficient one, but I understand this one better. -- Slaunger I have added your implementation to http://projects.scipy.org/numpy/ticket/1036 - is it ok with you to add the function eventually into arraysetops.py, under the numpy (BSD) license? cheers, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy array in iterable
Kim Hansen wrote: 2009/3/5 Robert Cimrman cimrm...@ntc.zcu.cz: I have added your implementation to http://projects.scipy.org/numpy/ticket/1036 - is it ok with you to add the function eventually into arraysetops.py, under the numpy (BSD) license? cheers, r. Yes, that would be fine with me. In fact that would be an honor! There is some formatting issue in the code you copied into the ticket... Cheers, Kim Great! It's a nice use case for return_inverse=True in unique1d(). I have fixed the formatting, but cannot remove the previous comment. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] intersect1d and setmember1d
Neil Crighton wrote: Robert Kern robert.kern at gmail.com writes: Do you mind if we just add you to the THANKS.txt file, and consider you as a NumPy Developer per the LICENSE.txt as having released that code under the numpy license? If we're dotting our i's and crossing our t's legally, that's a bit more straightforward (oddly enough). No, I don't mind having it released under the numpy licence. OK, I will tak care of including it - how should I proceed now? - has the workflow discussion settled somehow? r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] intersect1d and setmember1d
Robert Cimrman wrote: Neil Crighton wrote: Robert Kern robert.kern at gmail.com writes: Do you mind if we just add you to the THANKS.txt file, and consider you as a NumPy Developer per the LICENSE.txt as having released that code under the numpy license? If we're dotting our i's and crossing our t's legally, that's a bit more straightforward (oddly enough). No, I don't mind having it released under the numpy licence. OK, I will tak care of including it - how should I proceed now? - has the workflow discussion settled somehow? I have created http://projects.scipy.org/numpy/ticket/1036 - the patch will go there. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Faster way to generate a rotation matrix?
Jonathan Taylor wrote: Sorry.. obviously having some copy and paste trouble here. The message should be as follows: Hi, I am doing optimization on a vector of rotation angles tx,ty and tz using scipy.optimize.fmin. Unfortunately the function that I am optimizing needs the rotation matrix corresponding to this vector so it is getting constructed once for each iteration with new values. From profiling I can see that the function I am using to construct this rotation matrix is a bottleneck. I am currently using: def rotation(theta): tx,ty,tz = theta Rx = np.array([[1,0,0], [0, cos(tx), -sin(tx)], [0, sin(tx), cos(tx)]]) Ry = np.array([[cos(ty), 0, -sin(ty)], [0, 1, 0], [sin(ty), 0, cos(ty)]]) Rz = np.array([[cos(tz), -sin(tz), 0], [sin(tz), cos(tz), 0], [0,0,1]]) return np.dot(Rx, np.dot(Ry, Rz)) Is there a faster way to do this? Perhaps I can do this faster with a small cython module, but this might be overkill? Thanks for any help, Jonathan. An alternative to specifying the rotation by the three angles tx,ty and tz could be creating directly the rotation matrix given an axis and an angle: def make_axis_rotation_matrix(direction, angle): Create a rotation matrix corresponding to the rotation around a general axis by a specified angle. R = dd^T + cos(a) (I - dd^T) + sin(a) skew(d) Parameters: angle : float a direction : array d d = np.array(direction, dtype=np.float64) d /= np.linalg.norm(d) eye = np.eye(3, dtype=np.float64) ddt = np.outer(d, d) skew = np.array([[0, d[2], -d[1]], [-d[2], 0, d[0]], [d[1], -d[0],0]], dtype=np.float64) mtx = ddt + np.cos(angle) * (eye - ddt) + np.sin(angle) * skew return mtx r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] intersect1d and setmember1d
Neil wrote: mudit sharma mudit_19a at yahoo.com writes: intersect1d and setmember1d doesn't give expected results in case there are duplicate values in either array becuase it works by sorting data and substracting previous value. Is there an alternative in numpy to get indices of intersected values. In [31]: p nonzero(setmember1d(v1.Id, v2.Id))[0] [ 0 1 2 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 -- index 2 shouldn't be here look at the data below. 26 27 28 29] In [32]: p v1.Id[:10] [ 232. 232. 233. 233. 234. 234. 235. 235. 237. 237.] In [33]: p v2.Id[:10] [ 232. 232. 234. 234. 235. 235. 236. 236. 237. 237.] As far as I know there isn't an obvious way to get the functionality of setmember1d working on non-unique inputs. However, I've needed this operation quite a lot, so here's a function I wrote that does it. It's only a few times slower than numpy's setmember1d. You're welcome to use it. Hi Neil! I would like to add your function to arraysetops.py - is it ok? Just the name would be changed to setmember1d_nu, to follow the naming in the module (like intersect1d_nu). Thank you, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] intersect1d and setmember1d
Zachary Pincus wrote:
Hi,
intersect1d and setmember1d doesn't give expected results in case
there are duplicate values in either array becuase it works by
sorting data and substracting previous value. Is there an
alternative in numpy to get indices of intersected values.
From the docstring for setmember1d (and other set operations), you
are only supposed to pass it arrays with unique values (i.e. arrays
that represent sets in the mathematical sense):
print numpy.setmember1d.__doc__
Return a boolean array set True where first element is in second
array.
Boolean array is the shape of `ar1` containing True where the
elements of `ar1` are in `ar2` and False otherwise.
Use unique1d() to generate arrays with only unique elements to use as
inputs to this function. [...]
As stated, use unique1d to generate set-arrays from your input.
On the other hand, intersect1d is supposed to work with repeated
elements:
print numpy.intersect1d.__doc__
Intersection returning repeated or unique elements common to both
arrays.
Parameters -- ar1,ar2 : array_like Input arrays.
Returns --- out : ndarray, shape(N,) Sorted 1D array of common
elements with repeating elements.
See Also intersect1d_nu : Returns only unique common
elements. [...]
Do you have an example of intersect1d not working right? If so, what
version of numpy are you using (print numpy.version.version)?
Zach
Hi,
yes, many functions in arraysetops.py ('intersect1d', 'setxor1d',
'setmember1d', 'union1d', 'setdiff1d') were originally meant to work
with arrays of unique elements as inputs. I have just noticed, that the
docstring of intersect1d says that it works for non-unique arrays and
contains the following example:
np.intersect1d([1,3,3],[3,1,1])
array([1, 1, 3, 3])
I am not sure if this is a useful behaviour - does anybody uses this
feature (or better, side-effect)?
I would like to change the example to the usual use case:
In [9]: np.intersect1d([1,2,4,3],[3,1,5])
Out[9]: array([1, 3])
For arrays with non-unique elements, there is:
In [11]: np.intersect1d_nu([1,3,3],[3,1,1])
Out[11]: array([1, 3])
which just calls unique1d() for its arguments.
cheers,
r.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] concatenate trouble
Neal Becker wrote: What's the problem here? print np.concatenate (np.ones (10, dtype=complex), np.zeros (10, dtype=complex)) TypeError: only length-1 arrays can be converted to Python scalars You should enclose the arrays you concatenate into a tuple: np.concatenate((a,b)). r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] array manipulation
Nils Wagner wrote: Hi all, what is the best way to check if the entries (integers) of an array are stored in ascending order ? Hi Nils, Try np.alltrue( ar[1:] ar[:-1] ). r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] limit function
Neal Becker wrote: Is there a function to apply a limit to an array? I want to (efficiently) do: y = x if x limit, otherwise limit What about np.clip? r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique1d and asarray
Pierre GM wrote: On Jan 4, 2009, at 4:47 PM, Robert Kern wrote: On Sun, Jan 4, 2009 at 15:44, Pierre GM pgmdevl...@gmail.com wrote: If we used np.asanyarray instead, subclasses are recognized properly, the mask is recognized by argsort and the result correct. Is there a reason why we use np.asarray instead of np.asanyarray ? Probably not. So there wouldn't be any objections to make the switch ? We can wait a couple of days if anybody has a pb with that... There are probably other functions in arraysetops that could be fixed easily to work with masked arrays, feel free to do it if you like. I have never worked with the masked arrays, so the np.asarray problem had not come to my mind. Also, if you change np.asarray to np.asanyarray, add a corresponding test emplying the masked arrays to test_arraysetops.py, please. cheers thanks, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Find index of repeated numbers in array
Ross Williamson wrote: Hi Everyone I think I'm missing something really obvious but what I would like to do is extract the indexes from an array where a number matches - For example data = [0,1,2,960,5,6,960,7] I would like to know, for example the indices which match 960 - i.e. it would return 3 and 6 import numpy as np In[14]: np.where( np.array( data ) == 960 ) Out[14]: (array([3, 6]),) If you need to count all of the items, try something like np.histogram( data, np.max( data ) ) cheers, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] profiling line by line
Robert Kern wrote: On Fri, Sep 19, 2008 at 07:00, Robert Cimrman [EMAIL PROTECTED] wrote: Robert Kern wrote: Ah, found it. T_LONGLONG is a #define from structmember.h which is used to describe the types of attributes. Apparently, this was not added until Python 2.5. That particular member didn't actually need to be long long, so I've fixed that. Great, I will try it after it appears on the web page. Oops! It's now pushed. Thanks, it builds ok now! But then cProfile is also new in 2.5. - I should probably upgrade and stop bothering you :-) r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] profiling line by line
Robert Kern wrote: On Sun, Sep 21, 2008 at 02:09, Robert Cimrman [EMAIL PROTECTED] wrote: Robert Kern wrote: On Fri, Sep 19, 2008 at 07:00, Robert Cimrman [EMAIL PROTECTED] wrote: Robert Kern wrote: Ah, found it. T_LONGLONG is a #define from structmember.h which is used to describe the types of attributes. Apparently, this was not added until Python 2.5. That particular member didn't actually need to be long long, so I've fixed that. Great, I will try it after it appears on the web page. Oops! It's now pushed. Thanks, it builds ok now! But then cProfile is also new in 2.5. - I should probably upgrade and stop bothering you :-) Actually, if you pull again, I've just pushed my changes to remove the hard dependency on cProfile. Cool, then I have another one: $ ./kernprof.py -l pystone.py Traceback (most recent call last): File ./kernprof.py, line 173, in ? sys.exit(main(sys.argv)) File ./kernprof.py, line 138, in main import line_profiler File /home/share/software/packages/line_profiler-2ea515d0cbc1/line_profiler.py, line 9, in ? from _line_profiler import LineProfiler as CLineProfiler File _line_profiler.pyx, line 3, in _line_profiler (_line_profiler.c:2246) from cProfile import label ImportError: No module named cProfile Or do I call the profiler in a wrong way? r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] profiling line by line
Robert Kern wrote: On Sun, Sep 21, 2008 at 02:26, Robert Cimrman [EMAIL PROTECTED] wrote: Cool, then I have another one: $ ./kernprof.py -l pystone.py Traceback (most recent call last): File ./kernprof.py, line 173, in ? sys.exit(main(sys.argv)) File ./kernprof.py, line 138, in main import line_profiler File /home/share/software/packages/line_profiler-2ea515d0cbc1/line_profiler.py, line 9, in ? from _line_profiler import LineProfiler as CLineProfiler File _line_profiler.pyx, line 3, in _line_profiler (_line_profiler.c:2246) from cProfile import label ImportError: No module named cProfile Or do I call the profiler in a wrong way? Nope, I just had a brainfart and put the implementation of label() in the wrong file. Pull again. :-) The example works now in Python 2.4. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] profiling line by line
Robert Kern wrote: On Thu, Sep 18, 2008 at 06:01, Robert Cimrman [EMAIL PROTECTED] wrote: Hi Robert, Robert Kern wrote: On Mon, Sep 15, 2008 at 11:13, Arnar Flatberg [EMAIL PROTECTED] wrote: That would make me an extremely happy user, I've been looking for this for years! I can't imagine I'm the only one who profiles some hundred lines of code and ends up with 90% of total time in the dot-function For the time being, you can grab it here: http://www.enthought.com/~rkern/cgi-bin/hgwebdir.cgi/line_profiler/ It requires Cython and a C compiler to build. I'm still debating myself about the desired workflow for using it, but for now, it only profiles functions which you have registered with it. I have made the profiler work as a decorator to make this easy. E.g., many thanks for this! I have wanted to try out the profiler but failed to build it (changeset 6 0de294aa75bf): $ python setup.py install --root=/home/share/software/ running install running build running build_py creating build creating build/lib.linux-i686-2.4 copying line_profiler.py - build/lib.linux-i686-2.4 running build_ext cythoning _line_profiler.pyx to _line_profiler.c building '_line_profiler' extension creating build/temp.linux-i686-2.4 i486-pc-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -fPIC -I/usr/include/python2.4 -c -I/usr/include/python2.4 -c _line_profiler.c -o build/temp.linux-i686-2.4/_line_profiler.o _line_profiler.c:1614: error: 'T_LONGLONG' undeclared here (not in a function) error: command 'i486-pc-linux-gnu-gcc' failed with exit status 1 I have cython-0.9.8.1 and GCC 4.1.2, 32-bit machine. It uses the #define'd macro PY_LONG_LONG. Go through your Python headers to see what this gets expanded to. I have Python 2.4.4 in pyconfig.h #define HAVE_LONG_LONG 1 in pyport.h: #ifdef HAVE_LONG_LONG #ifndef PY_LONG_LONG #define PY_LONG_LONG long long #endif #endif /* HAVE_LONG_LONG */ so it seems compatible with 'ctypedef long long PY_LONG_LONG' r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] profiling line by line
Ondrej Certik wrote: On Thu, Sep 18, 2008 at 4:12 PM, Ryan May [EMAIL PROTECTED] wrote: Ondrej Certik wrote: On Thu, Sep 18, 2008 at 1:01 PM, Robert Cimrman [EMAIL PROTECTED] wrote: It requires Cython and a C compiler to build. I'm still debating myself about the desired workflow for using it, but for now, it only profiles functions which you have registered with it. I have made the profiler work as a decorator to make this easy. E.g., many thanks for this! I have wanted to try out the profiler but failed to build it (changeset 6 0de294aa75bf): $ python setup.py install --root=/home/share/software/ running install running build running build_py creating build creating build/lib.linux-i686-2.4 copying line_profiler.py - build/lib.linux-i686-2.4 running build_ext cythoning _line_profiler.pyx to _line_profiler.c building '_line_profiler' extension creating build/temp.linux-i686-2.4 i486-pc-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -fPIC -I/usr/include/python2.4 -c -I/usr/include/python2.4 -c _line_profiler.c -o build/temp.linux-i686-2.4/_line_profiler.o _line_profiler.c:1614: error: 'T_LONGLONG' undeclared here (not in a function) error: command 'i486-pc-linux-gnu-gcc' failed with exit status 1 I have cython-0.9.8.1 and GCC 4.1.2, 32-bit machine. I am telling you all the time Robert to use Debian that it just works and you say, no no, gentoo is the best. :) And what's wrong with that? :) Once you get over the learning curve, Gentoo works just fine. Must be Robert K.'s fault. :) Well, I think if Robert C. hasn't yet get over the learning curve after so many years of hard work, maybe the learning curve is too steep. :) This is most probably not related to Gentoo at all and certainly not related to me knowing Gentoo or not :) (and no, learning Gentoo is not that hard.) r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] profiling line by line
Robert Kern wrote: Ah, found it. T_LONGLONG is a #define from structmember.h which is used to describe the types of attributes. Apparently, this was not added until Python 2.5. That particular member didn't actually need to be long long, so I've fixed that. Great, I will try it after it appears on the web page. Thank you, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique1d returning indices
Stéfan van der Walt wrote: 2008/8/13 Robert Cimrman [EMAIL PROTECTED]: Yeah, that's why I think not many people used the extra return anyway. I will do as you say unless somebody steps in. ... but not before August 25, as I am about to leave on holidays and have not managed to do it yet. I do not want to mess with the SVN now as I would not be able to follow it. If you think the patch is ok, and have time, then go ahead :) Thanks, Robert. This has been merged in r5639. Nice, thank you. After I return I will go through the other arraysetops functions and add return_index-like flags when appropriate. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique1d returning indices
Robert Cimrman wrote: Stéfan van der Walt wrote: 2008/8/11 Robert Cimrman [EMAIL PROTECTED]: Note also that the order of outputs has changed (previously unique1d() returned (i, b) for return_index=True). Does this not constitute an API change? It does. Are there many users of unique1d( a, return_index=True ) out there? The new way is consistent with matlab, and IMHO more logical - the additional return values are appended to the returned tuple rather than prepended. It was my mistake to introduce the previous order (do not recall the motivation anymore :]). The new way is definitely more consistent -- I think one could almost call its current behaviour a bug. Unless there are objections, let's follow the standard route: put a deprecation warning in place for one release and remove it in the next. Yeah, that's why I think not many people used the extra return anyway. I will do as you say unless somebody steps in. ... but not before August 25, as I am about to leave on holidays and have not managed to do it yet. I do not want to mess with the SVN now as I would not be able to follow it. If you think the patch is ok, and have time, then go ahead :) r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] unique1d returning indices
Stéfan van der Walt wrote: 2008/8/11 Robert Cimrman [EMAIL PROTECTED]: Note also that the order of outputs has changed (previously unique1d() returned (i, b) for return_index=True). Does this not constitute an API change? It does. Are there many users of unique1d( a, return_index=True ) out there? The new way is consistent with matlab, and IMHO more logical - the additional return values are appended to the returned tuple rather than prepended. It was my mistake to introduce the previous order (do not recall the motivation anymore :]). The new way is definitely more consistent -- I think one could almost call its current behaviour a bug. Unless there are objections, let's follow the standard route: put a deprecation warning in place for one release and remove it in the next. Yeah, that's why I think not many people used the extra return anyway. I will do as you say unless somebody steps in. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] member1d and unique elements
Hi Greg, Greg Novak wrote: Argh. I could swear that yesterday I typed test cases just like the one you provide, and it behaved correctly. Nevertheless, it clearly fails in spite of my memory, so attached is a version which I believe gives the correct behavior. It looks ok now, although I did not check very carefully. Just make sure that it works for empty input arrays (all combinations). I would also set the default to handle_dupes to False to preserve previous functionality. I suggest you refresh your numpy repository from SVN - now np. prefix is used instead of nm. within the module. Thank you! r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] unique1d returning indices
Hi, due to popular demand, I have updated unique1d() to optionally return both kinds of indices: In [3]: b, i, j = nm.unique1d( a, return_index=True, return_inverse=True ) In [4]: a Out[4]: array([1, 1, 8, 3, 3, 5, 4]) In [6]: b Out[6]: array([1, 3, 4, 5, 8]) In [7]: a[i] Out[7]: array([1, 3, 4, 5, 8]) In [8]: b[j] Out[8]: array([1, 1, 8, 3, 3, 5, 4]) In [9]: a Out[9]: array([1, 1, 8, 3, 3, 5, 4]) Could someone needing this try the attached code? If it is ok I will update the tests, the docstring and commit it. Note also that the order of outputs has changed (previously unique1d() returned (i, b) for return_index=True). cheers, r. def unique1d(ar1, return_index=False, return_inverse=False): Find the unique elements of an array. Parameters -- ar1 : array-like This array will be flattened if it is not already 1-D. return_index : bool, optional If True, also return the indices against `ar1` that result in the unique array. return_inverse : bool, optional If True, also return the indices against the unique array that result in `ar1`. Returns --- unique : ndarray The unique values. unique_indices : ndarray, optional The indices of the unique values. Only provided if `return_index` is True. unique_inverse : ndarray, optional The indices to reconstruct the original array. Only provided if `return_inverse` is True. See Also numpy.lib.arraysetops : Module with a number of other functions for performing set operations on arrays. Examples np.unique1d([1, 1, 2, 2, 3, 3]) array([1, 2, 3]) a = np.array([[1, 1], [2, 3]]) np.unique1d(a) array([1, 2, 3]) ar = np.asarray(ar1).flatten() if ar.size == 0: if return_inverse and return_index: return ar, np.empty(0, np.bool), np.empty(0, np.bool) elif return_inverse or return_index: return ar, np.empty(0, np.bool) else: return ar if return_inverse or return_index: perm = ar.argsort() aux = ar[perm] flag = np.concatenate( ([True], aux[1:] != aux[:-1]) ) if return_inverse: iflag = np.cumsum( flag ) - 1 iperm = perm.argsort() if return_index: return aux[flag], perm[flag], iflag[iperm] else: return aux[flag], iflag[iperm] else: return aux[flag], perm[flag] else: ar.sort() flag = np.concatenate( ([True], ar[1:] != ar[:-1]) ) return ar[flag] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] member1d and unique elements
Greg Novak wrote: I have two arrays of integers, and would like to know _where_ they have elements in common, not just _which_ elements are in common. This is because the entries in the integer array are aligned with other arrays. This seems very close to what member1d advertises as its function. However, member1d says that it expects arrays with only unique elements. First of all, my desired operation is well-posed: I'd like f(ar1, ar2) to return something in the shape of ar1 with True if the value at that position appears anywhere in ar2 (regardless of duplication) and False otherwise. So I looked at the code and have two questions: 1) What is this code trying to achieve? aux = perm[ii+1] perm[ii+1] = perm[ii] perm[ii] = aux Here perm is the stable argsort of the two concatenated arguments: perm = concatenate((ar1, ar2)).argsort(kind='mergesort'). arr is the array of combined inputs in sorted order: arr = concatenate((ar1, ar2))[perm] and ii is a list of indices into arr where the value of arr is equal to the next value in the array (arr[ii] == arr[ii+1]) _and_ arr[ii] came from the _second_ input (ar2). Now, this last bit (looking for elements of arr that are equal and both came from the second array) is clearly trying to deal with duplication, which is why I'm interested... So, the code snippet is trying to swap perm[ii+1] with perm[ii], but I don't see why. Furthermore, there are funny results if a value is duplicated three times, not just twice -- perm is no longer a permutation vector. Eg, member1d([1], [2,2,2]) results perm=[0,1,2,3] and ii=[1,2] before the above snippet, and the above snippet makes perm into [0,2,3,2] I've commented those three lines, and I've never seen any changes to the output of member1d. The new value of perm is used to compute the expression: perm.argsort(kind='mergesort')[:len( ar1 )], but the changes to that expression as a result of the above three lines are always at the high end of the array, which is sliced off by the last [:len(ar1)]. Finally, my second question is: 2) Does anyone have a test case where member1d fails as a result of duplicates in the input? So far I haven't found any, with the above lines commented or not. Upon reflection and review of the changelog, another theory occurs to me: member1d did not originally use a stable sort. What I've written above for interpretation of the value ii (indicates duplication within ar2) is true for a stable sort, but for an unstable sort the same condition has the interpretation that ii holds the values where the sorting algorithm swapped the order of equal values unstably. Then the code snippet in question 1) looks like an attempt to swap those values in the permutation array to make the sort stable again. The attempt would fail if there was duplication in either array. So, I would propose deleting those three lines (since they seem to be a non-functional relic) and declaring in the docstring that member1d doesn't require unique elements. Also, if this is correct, then the function simplifies considerably since several values don't need to be computed anymore: def setmember1d( ar1, ar2 ): ar = nm.concatenate( (ar1, ar2 ) ) perm = ar.argsort(kind='mergesort') aux = ar[perm] flag = nm.concatenate( (aux[1:] == aux[:-1], [False] ) ) indx = perm.argsort(kind='mergesort')[:len( ar1 )] return flag[indx] Corrections to the above are welcome since I'm going to start using member1d without regard for uniqueness, and I'd like to know if I'm making a big mistake... Hi Greg, I do not have much time to investigate it in detail right now, but it does not work for repeated entries in ar1: In [14]: nm.setmember1d( [1,2,3,2], [1, 3] ) Out[14]: array([ True, True, True, False], dtype=bool) thanks for trying to enhance arraysetops! r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 00.46.02
I am pleased announce the release of SfePy 00.46.02. SfePy is a finite element analysis software in Python, based primarily on Numpy and SciPy. Mailing lists, issue tracking, mercurial repository: http://sfepy.org Home page: http://sfepy.kme.zcu.cz Major improvements: - alternative short syntax for specifying essential boundary conditions, variables and regions - manufactured solutions tests: - SymPy support - site configuration now via script/config.py + site_cfg.py - new solvers - new terms For more information on this release, see http://sfepy.googlecode.com/svn/web/releases/004602_RELEASE_NOTES.txt If you happen to come to Leipzig for EuroSciPy 2008, see you there! Best regards, Robert Cimrman SfePy developers ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] set_printoptions - floating point format option?
Hi, I need to display some numpy arrays in mantissa+exponent format (e.g. '%.2e' using C syntax). In numpy.set_printoptions(), there is currently only 'precision' option, which does not allow this. What about having an option related to 'precision', named possibly 'float_format', with the following function: :Parameters: format : string Format string for floating point output, has precedence over 'precision' (default '%.8f'). r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SfePy 00.41.03
Greetings, I'm pleased to announce the release 00.41.03 of SfePy (formerly SFE) SfePy is a finite element analysis software in Python, based primarily on Numpy and SciPy. Mailing lists, issue tracking, mercurial repository: http://code.google.com/p/sfepy/ Home page: http://sfepy.kme.zcu.cz Major improvements: - works on 64 bits - support for various mesh formats - Schroedinger equation solver - see http://code.google.com/p/sfepy/wiki/Examples - new solvers: - generic time-dependent problem solver - pysparse, symeig, scipy-based eigenproblem solvers - scipy-based iterative solvers - many new terms For information on this release, see http://sfepy.googlecode.com/svn/web/releases/004103_RELEASE_NOTES.txt Best regards, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SFE-00.35.01
Let me announce SFE-00.35.01, bringing per term integration - now each term can use its own quadrature points. This is a major change at the heart of the code - some parts may not work as all terms were not migrated yet to the new framework. All test examples work, though, as well as acoustic band gaps. See http://ui505p06-mbs.ntc.zcu.cz/sfe . SFE is a finite element analysis software written almost entirely in Python. The code is released under BSD license. best regards, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] documentation generator based on pyparsing
Nils Wagner wrote: Thank you for your note. It works fine for me with python2.5. However python2.3 results in ./gendocs.py -m 'scipy.linsolve.umfpack' Traceback (most recent call last): File ./gendocs.py, line 261, in ? main() File ./gendocs.py, line 207, in main default = 1, help = help['page'] ) File /usr/lib64/python2.3/optparse.py, line 820, in add_option option = self.option_class(*args, **kwargs) File /usr/lib64/python2.3/optparse.py, line 430, in __init__ checker(self) File /usr/lib64/python2.3/optparse.py, line 499, in _check_type raise OptionError(invalid option type: %r % self.type, self) optparse.OptionError: option -p/--page: invalid option type: type 'int' How can I resolve this problem ? In gendocs.py, line 205, try to replace 'type = int' with 'type = int': parser.add_option( -p, --page, metavar = 'page', type = int, action = store, dest = page, default = 1, help = help['page'] ) thanks for reporting bugs, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] documentation generator based on pyparsing
Hi Nils, Nils Wagner wrote: The output of ./gendocs.py -m 'scipy.linsolve.umfpack' differs from your example output (available at http://scipy.org/Generate_Documentation) I had to update the umfpack info.py file (where the module docstring is) to conform the documentation standards. The gendocs.py relies on that, so use, please, the newest SVN version of scipy - it should work with rev. 3601 and later. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] documentation generator based on pyparsing
Hi, At http://scipy.org/Generate_Documentation you can find a very small documentation generator for NumPy/SciPy modules based on pyparsing package (by Paul McGuire). I am not sure if this belongs to where I put it, so feel free to (re)move the page as needed. I hope it might be interesting for you. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy : your experiences?
Rahul Garg wrote: It would be awesome if you guys could respond to some of the following questions : a) Can you guys tell me briefly about the kind of problems you are tackling with numpy and scipy? I am using both numpy and scipy to solve PDEs in the context of finite element method (elasticity, porous media, ...). b) Have you ever felt that numpy/scipy was slow and had to switch to C/C++/Fortran? I use bits of C code wrapped by SWIG either to address real bottle-necks (FE assembling, element matrix computations) or due to the fact that I have lots of legacy C code that I reuse from my previous projects. c) Do you use any form of parallel processing? Multicores? SMPs? Clusters? If yes how did u utilize them? If I ever get myself to start, I will use petsc4py. If you feel its not relevant to the list .. feel free to email me personally. I would be very interested in talking about these issues. It is really interesting to see what others do. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: SFE-00.31.06 release
I am happy to announce the version 00.31.06 of SFE, featuring acoustic band gaps computation, rigid body motion constraints, new solver classes and reorganization, and regular bug fixes and updates, see http://ui505p06-mbs.ntc.zcu.cz/sfe. SFE is a finite element analysis software written almost entirely in Python. This version is released under BSD license. best wishes, r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Naming a slice index?
Eagle Jones wrote: New to python and numpy; hopefully I'm missing something obvious. I'd like to be able to slice an array with a name. For example: _T = 6:10 _T = slice( 6, 10 ) ... ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Count the occurrence of a certain integer in a list of integers
Nils Wagner wrote: Hi all, I have a list of integer numbers. The entries can vary between 0 and 19. How can I count the occurrence of any number. Consider data [9, 6, 9, 6, 7, 9, 9, 10, 7, 9, 9, 6, 7, 9, 8, 8, 11, 9, 6, 7, 10, 9, 7, 9, 7, 8, 9, 8, 7, 9] Is there a better way than using, e.g. shape(where(array(data)==10))[1] 2 to compute the occurrence of 10 in the list which is 2 in this case ? Your way is ok if you want to count just a few numbers. If you want all, you may sort the array and use searchorted: b = sort( a ) count = searchsorted( b, 7, side = 'right' ) - searchsorted( b, 7, side = 'left' ) r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Tuning sparse stuff in NumPy
David Koch wrote: On 3/27/07, Robert Cimrman [EMAIL PROTECTED] wrote: ok. now which version of scipy (scipy.__version__) do you use (you may have posted it, but I missed it)? Not so long ago, there was an effort by Nathan Bell and others reimplementing sparsetools + scipy.sparse to get better usability and performance. My (almost latest) version is 0.5.3.dev2860. My version is 0.5.2, the latest official version available for download, right? I see. Now while it is desirable and fine in majority of cases to install stable version of software, with numpy and scipy it is different :-), as the development is very fast. You should try the 'bleeding edge' versions via SVN, as many issues you see could have been already fixed/improved. Thanks to the extensive unit testing, the latest version usually (read: 99.9%) works better than the stable one. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Tuning sparse stuff in NumPy
David Koch wrote: On 3/26/07, Robert Cimrman [EMAIL PROTECTED] wrote: Could you be more specific on which type of the sparse matrix storage did you use? Hi Robert, I used csc_matrix. OK, good. Would you mind measuring csc * csr, csc * csc, csr * csc and csr * csr? I am curious how this will compare. r. ps: this thread might be more appropriate for scipy-user or scipy-dev... ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab - NumPy translation and indexing
David Koch wrote: Hi, so one thing I came across now is the following, very simple: Matlab: A = [] while A = [A some_scalar_value] end In Python, I tried: A = empty((0,0)) while A = concatenate((A, array([someScalarValue])), 1) end which returns an error since the shape of the empty A does not match the vector I want to concatenate with. Any way to get around this without having to use some obscure if exist A clause. I am still stuck in my Matlab ways - hoping to get rid of them though. a = empty( (0,) ) ... I would bet that using a regular Python list would be faster in this case than growing the numpy array. a = [] while ... a.append( scalar ) a = array( a ) r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Managing Rolling Data
Alexander Michael wrote: I'm new to numpy and looking for advice on setting up and managing array data for my particular problem. I'm collecting observations of P properties for N objects over a rolling horizon of H sample times. I could conceptually store the data in three-dimensional array with shape (N,P,H) that would allow me to easily (and efficiently with strided slices) compute the statistics over both N and H that I am interested in. This is great, but the rub is that H, an interval of T, is a rolling horizon. T is to large to fit in memory, so I need to load up H, perform my calculations, pop the oldest N x P slice and push the newest N x P slice into the data cube. What's the best way to do this that will maintain fast computations along the one-dimensional slices over N and H? Is there a commonly accepted idiom? Fundamentally, I see two solutions. The first would be to essentially perform a memcpy to propagate the data. The second would be to manage the N x P slices as H discontiguous memory blocks and merely reorder the pointers with each new sample. Can I do either of these with numpy? The 'pointers reordering' can be very nicely done via deque from collections module - it behaves like a list, but has some additional methods like rotate(). I have used it successfully as a circular buffer for numpy arrays, but I can see that you need an efficient slicing over H, so a contiguous memory would be better for you IMHO. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] getting indices for array positions
Christian Meesters wrote: Hi This questions might seem stupid, but I didn't get a clever solution myself, or found one in the archives, the cookbook, etc. . If I overlooked something, please give a pointer. Well, if I have an 1D array like [ 0. , 0.1, 0.2, 0.3, 0.4, 0.5] ,a scalar like 0.122 and want to retrieve the index postion of the closest value of the scalar in the array: Is there any fast method to get this? Try searchsorted. r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] getting indices for array positions
Christian Meesters wrote: Try searchsorted. Thanks, but that doesn't work. Sorry, if my question wasn't clear. To illustrate the requirement: For instance: a array([ 0. , 0.1, 0.2, 0.3, 0.4]) # should be 1 ... a.searchsorted(0.11) 2 # should be 2 ... a.searchsorted(0.16) 2 I could correct for one index position, of course, but I still have the requirement to get the index of the item with the closest value to the key. Since searchsorted returns the index of the first item in a that is = or the key, it can't make the distinction between 0.1 and 0.2 as I would like to have. I see. But it gives you the index (say 'ii') of the first item that is grater than your scalar - then you just have to compare your scalar with a[ii] and a[ii-1] and choose whichever is closer, no? r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] setmember1d memory leak?
Charles R Harris wrote: On 1/24/07, Charles R Harris [EMAIL PROTECTED] wrote: On 1/24/07, Robert Cimrman [EMAIL PROTECTED] wrote: Robert Kern wrote: Robert Cimrman wrote: Or you could just call unique1d prior to your call to setmember1d - it was meant to be used that way... you would not loose much speed that way, IMHO. But that doesn't do what they want. They want a function that gives the mask against their original array of the elements that are in the other array. The result of setmember1d(unique1d(ar1), unique1d(ar2)) snip For instance In [7]: def countmembers(a1, a2) : ...: a = sort(a2) ...: il = a.searchsorted(a1, side='l') ...: ir = a.searchsorted(a1, side='r') ...: return ir - il ...: In [8]: a2 = random.randint(0,10,(100,)) In [9]: a1 = arange(11) In [11]: a2 = random.randint(0,5,(100,)) In [12]: a1 = arange(10) In [13]: countmembers(a1,a2) Out[13]: array([16, 28, 16, 25, 15, 0, 0, 0, 0, 0]) The subtraction can be replaced by != to get a boolean mask. It looks good! Isn't it faster than setmember1d for unique input arrays? I do not like setmember1d much (it is long unlike other functions in arraysetops and looks clumsy to me now and I do not understand it anymore...), so feel free to replace it. BTW. setmember1d gives me the same mask as countmembers for several non-unique inputs I tried... r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
