[Numpy-discussion] conversion to 1D array of specified dtype
Hello list, What is the easiest way to convert a function argument to at least a 1D array of specified dtype? atleast_1d(3,dtype='d') doesn't work (numpy 1.3.0) array(atleast_1d(3),dtype='d') works but seems cumbersome Thanks, Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Boolean arrays
Nathaniel Smith njs at pobox.com writes: On Fri, Aug 27, 2010 at 1:35 PM, Robert Kern robert.kern at gmail.com wrote: As valid gets larger, in1d() will catch up but for smallish sizes of valid, which I suspect given the non-numeric nature of the OP's (Hi, Brett!) request, kern_in() is usually better. Oh well, I was just guessing based on algorithmic properties. Sounds like there might be some optimizations possible to in1d then, if anyone had a reason to care . Ideally, I would like in1d to always be the right answer to this problem. It should be easy to put in an if statement to switch to a kern_in()-type function in the case of large ar1 but small ar2. I will do some timing tests and make a patch. Incidentally, the timing tests done when in1d was introduced only considered the case when len(ar1) = len(ar2). In this case the current in_1d is pretty much always faster than kern_in(). Neil ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy histogram normed=True (bug / confusing behavior)
On Tue, Aug 31, 2010 at 3:44 AM, David Huard david.hu...@gmail.com wrote:
I just added a warning alerting concerned users (r8674), so this takes care
of the bug fix and Nils wish to avoid a silent change in behavior. These two
changes could be included in 1.5 if Ralf feels this is worthwhile.
That looks like a reasonable solution. I haven't got a strong opinion on
whether or not to change the 'normed' keyword to 'density'.
Looking at the changes, I don't think that is the right way to do the
filtering in the tests. resetwarnings() removes all filters including the
ones previously set by users, and should therefore not be used. Better to
either raise a specific warning and filter on that, or to filter on the
message content with:
warnings.filterwarnings('ignore' , message=This release of NumPy fixes
a normalization bug in histogram).
I found one more place where resetwarnings() is used, in
test_arraysetops.py, I'll change that in trunk. Related problem there is
that the warning in warnings.warn is not a DeprecationWarning.
The above problem is easy to fix, but in any case it's too late to go into
1.5.0 - I'll tag the final release tonight.
Cheers,
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] conversion to 1D array of specified dtype
On Tue, Aug 31, 2010 at 3:57 AM, Mark Bakker mark...@gmail.com wrote:
Hello list,
What is the easiest way to convert a function argument to at least a 1D
array of specified dtype?
atleast_1d(3,dtype='d') doesn't work (numpy 1.3.0)
array(atleast_1d(3),dtype='d') works but seems cumbersome
atleast_1d(d).astype('d')
Ryan
--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy histogram normed=True (bug / confusing behavior)
On Tue, Aug 31, 2010 at 7:02 AM, Ralf Gommers
ralf.gomm...@googlemail.comwrote:
On Tue, Aug 31, 2010 at 3:44 AM, David Huard david.hu...@gmail.comwrote:
I just added a warning alerting concerned users (r8674), so this takes
care of the bug fix and Nils wish to avoid a silent change in behavior.
These two changes could be included in 1.5 if Ralf feels this is
worthwhile.
That looks like a reasonable solution. I haven't got a strong opinion on
whether or not to change the 'normed' keyword to 'density'.
Looking at the changes, I don't think that is the right way to do the
filtering in the tests. resetwarnings() removes all filters including the
ones previously set by users, and should therefore not be used. Better to
either raise a specific warning and filter on that, or to filter on the
message content with:
warnings.filterwarnings('ignore' , message=This release of NumPy fixes
a normalization bug in histogram).
I found one more place where resetwarnings() is used, in
test_arraysetops.py, I'll change that in trunk. Related problem there is
that the warning in warnings.warn is not a DeprecationWarning.
The above problem is easy to fix, but in any case it's too late to go into
1.5.0 - I'll tag the final release tonight.
Ralf,
test_function_base and test_arraysetops now do not use resetwarnings. What I
did is added a warning filter and popped it out of the filters list
afterwards. Is this OK ?
In other tests, what is done is rather
warnings.simplefilter('ignore', DeprecationWarning)
test_function()
warnings.simplefilter('default', DeprecationWarning)
but that will also override any user-defined setup, no ?
David
Cheers,
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy histogram normed=True (bug / confusing behavior)
On Tue, Aug 31, 2010 at 10:13 PM, David Huard david.hu...@gmail.com wrote:
On Tue, Aug 31, 2010 at 7:02 AM, Ralf Gommers ralf.gomm...@googlemail.com
wrote:
On Tue, Aug 31, 2010 at 3:44 AM, David Huard david.hu...@gmail.comwrote:
I just added a warning alerting concerned users (r8674), so this takes
care of the bug fix and Nils wish to avoid a silent change in behavior.
These two changes could be included in 1.5 if Ralf feels this is
worthwhile.
That looks like a reasonable solution. I haven't got a strong opinion on
whether or not to change the 'normed' keyword to 'density'.
Looking at the changes, I don't think that is the right way to do the
filtering in the tests. resetwarnings() removes all filters including the
ones previously set by users, and should therefore not be used. Better to
either raise a specific warning and filter on that, or to filter on the
message content with:
warnings.filterwarnings('ignore' , message=This release of NumPy
fixes a normalization bug in histogram).
I found one more place where resetwarnings() is used, in
test_arraysetops.py, I'll change that in trunk. Related problem there is
that the warning in warnings.warn is not a DeprecationWarning.
The above problem is easy to fix, but in any case it's too late to go into
1.5.0 - I'll tag the final release tonight.
Ralf,
test_function_base and test_arraysetops now do not use resetwarnings. What
I did is added a warning filter and popped it out of the filters list
afterwards. Is this OK ?
That looks like the best solution to me (besides the catch_warnings context
manager, which we can't use).
In other tests, what is done is rather
warnings.simplefilter('ignore', DeprecationWarning)
test_function()
warnings.simplefilter('default', DeprecationWarning)
but that will also override any user-defined setup, no ?
Yes, it's less invasive than resetwarnings() but could also be annoying for
some users. Would be good to do this in a consistent way for all warnings
and tests, and describe that in
http://projects.scipy.org/numpy/wiki/ApiDeprecation.
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: NumPy 1.5.0 release
I am pleased to announce the availability of NumPy 1.5.0. This is the first NumPy release to include support for Python 3, as well as for Python 2.7. Binaries, sources, documentation and release notes can be found at https://sourceforge.net/projects/numpy/files/. Thank you to everyone who contributed to this release. Enjoy, the Numpy developers = NumPy 1.5.0 Release Notes = Highlights == Python 3 compatibility -- This is the first NumPy release which is compatible with Python 3. Support for Python 3 and Python 2 is done from a single code base. Extensive notes on changes can be found at `http://projects.scipy.org/numpy/browser/trunk/doc/Py3K.txt`_. Note that the Numpy testing framework relies on nose, which does not have a Python 3 compatible release yet. A working Python 3 branch of nose can be found at `http://bitbucket.org/jpellerin/nose3/`_ however. Porting of SciPy to Python 3 is expected to be completed soon. :pep:`3118` compatibility - The new buffer protocol described by PEP 3118 is fully supported in this version of Numpy. On Python versions = 2.6 Numpy arrays expose the buffer interface, and array(), asarray() and other functions accept new-style buffers as input. New features Warning on casting complex to real -- Numpy now emits a `numpy.ComplexWarning` when a complex number is cast into a real number. For example: x = np.array([1,2,3]) x[:2] = np.array([1+2j, 1-2j]) ComplexWarning: Casting complex values to real discards the imaginary part The cast indeed discards the imaginary part, and this may not be the intended behavior in all cases, hence the warning. This warning can be turned off in the standard way: import warnings warnings.simplefilter(ignore, np.ComplexWarning) Dot method for ndarrays --- Ndarrays now have the dot product also as a method, which allows writing chains of matrix products as a.dot(b).dot(c) instead of the longer alternative np.dot(a, np.dot(b, c)) linalg.slogdet function --- The slogdet function returns the sign and logarithm of the determinant of a matrix. Because the determinant may involve the product of many small/large values, the result is often more accurate than that obtained by simple multiplication. new header -- The new header file ndarraytypes.h contains the symbols from ndarrayobject.h that do not depend on the PY_ARRAY_UNIQUE_SYMBOL and NO_IMPORT/_ARRAY macros. Broadly, these symbols are types, typedefs, and enumerations; the array function calls are left in ndarrayobject.h. This allows users to include array-related types and enumerations without needing to concern themselves with the macro expansions and their side- effects. Changes === polynomial.polynomial - * The polyint and polyder functions now check that the specified number integrations or derivations is a non-negative integer. The number 0 is a valid value for both functions. * A degree method has been added to the Polynomial class. * A trimdeg method has been added to the Polynomial class. It operates like truncate except that the argument is the desired degree of the result, not the number of coefficients. * Polynomial.fit now uses None as the default domain for the fit. The default Polynomial domain can be specified by using [] as the domain value. * Weights can be used in both polyfit and Polynomial.fit * A linspace method has been added to the Polynomial class to ease plotting. * The polymulx function was added. polynomial.chebyshev * The chebint and chebder functions now check that the specified number integrations or derivations is a non-negative integer. The number 0 is a valid value for both functions. * A degree method has been added to the Chebyshev class. * A trimdeg method has been added to the Chebyshev class. It operates like truncate except that the argument is the desired degree of the result, not the number of coefficients. * Chebyshev.fit now uses None as the default domain for the fit. The default Chebyshev domain can be specified by using [] as the domain value. * Weights can be used in both chebfit and Chebyshev.fit * A linspace method has been added to the Chebyshev class to ease plotting. * The chebmulx function was added. * Added functions for the Chebyshev points of the first and second kind. histogram - After a two years transition period, the old behavior of the histogram function has been phased out, and the new keyword has been removed. correlate - The old behavior of correlate was deprecated in 1.4.0, the new behavior (the usual definition for cross-correlation) is now the default. Checksums = 738572dd3e5d4cd89e98c76cc3f162a9 release/installers/numpy-1.5.0-py2.5-python.org.dmg f58ebc1840974cf2ef8b4f41516dc288
[Numpy-discussion] comparison between arrays of strings and numerical types
Hi, I find this a bit odd: In [18]: np.array(['a','b','c','d']) 'a' Out[18]: array([False, True, True, True], dtype=bool) In [19]: np.array(['a','b','c','d']) 4 Out[19]: True In [20]: np.array(['a','b','c','d']) 4.5 Out[20]: True Is that right? I was expecting an element-wise comparison, but it returns a single truth value. Cheers, Ernest ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] comparison between arrays of strings and numerical types
2010/8/31 Ernest Adrogué eadro...@gmx.net: Hi, I find this a bit odd: In [18]: np.array(['a','b','c','d']) 'a' Out[18]: array([False, True, True, True], dtype=bool) In [19]: np.array(['a','b','c','d']) 4 Out[19]: True In [20]: np.array(['a','b','c','d']) 4.5 Out[20]: True Is that right? I was expecting an element-wise comparison, but it returns a single truth value. Here's a recent discussion of the issue: http://www.mail-archive.com/numpy-discussion@scipy.org/msg27051.html ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: NumPy 1.5.0 release
On Tue, Aug 31, 2010 at 10:18 AM, Ralf Gommers ralf.gomm...@googlemail.comwrote: I am pleased to announce the availability of NumPy 1.5.0. This is the first NumPy release to include support for Python 3, as well as for Python 2.7. Binaries, sources, documentation and release notes can be found at https://sourceforge.net/projects/numpy/files/. Thank you to everyone who contributed to this release. Enjoy, the Numpy developers = NumPy 1.5.0 Release Notes = Highlights == Python 3 compatibility -- This is the first NumPy release which is compatible with Python 3. Support for Python 3 and Python 2 is done from a single code base. Extensive notes on changes can be found at `http://projects.scipy.org/numpy/browser/trunk/doc/Py3K.txt`_. Note that the Numpy testing framework relies on nose, which does not have a Python 3 compatible release yet. A working Python 3 branch of nose can be found at `http://bitbucket.org/jpellerin/nose3/`_ however. Porting of SciPy to Python 3 is expected to be completed soon. :pep:`3118` compatibility - The new buffer protocol described by PEP 3118 is fully supported in this version of Numpy. On Python versions = 2.6 Numpy arrays expose the buffer interface, and array(), asarray() and other functions accept new-style buffers as input. New features Warning on casting complex to real -- Numpy now emits a `numpy.ComplexWarning` when a complex number is cast into a real number. For example: x = np.array([1,2,3]) x[:2] = np.array([1+2j, 1-2j]) ComplexWarning: Casting complex values to real discards the imaginary part The cast indeed discards the imaginary part, and this may not be the intended behavior in all cases, hence the warning. This warning can be turned off in the standard way: import warnings warnings.simplefilter(ignore, np.ComplexWarning) Dot method for ndarrays --- Ndarrays now have the dot product also as a method, which allows writing chains of matrix products as a.dot(b).dot(c) instead of the longer alternative np.dot(a, np.dot(b, c)) linalg.slogdet function --- The slogdet function returns the sign and logarithm of the determinant of a matrix. Because the determinant may involve the product of many small/large values, the result is often more accurate than that obtained by simple multiplication. new header -- The new header file ndarraytypes.h contains the symbols from ndarrayobject.h that do not depend on the PY_ARRAY_UNIQUE_SYMBOL and NO_IMPORT/_ARRAY macros. Broadly, these symbols are types, typedefs, and enumerations; the array function calls are left in ndarrayobject.h. This allows users to include array-related types and enumerations without needing to concern themselves with the macro expansions and their side- effects. Changes === polynomial.polynomial - * The polyint and polyder functions now check that the specified number integrations or derivations is a non-negative integer. The number 0 is a valid value for both functions. * A degree method has been added to the Polynomial class. * A trimdeg method has been added to the Polynomial class. It operates like truncate except that the argument is the desired degree of the result, not the number of coefficients. * Polynomial.fit now uses None as the default domain for the fit. The default Polynomial domain can be specified by using [] as the domain value. * Weights can be used in both polyfit and Polynomial.fit * A linspace method has been added to the Polynomial class to ease plotting. * The polymulx function was added. polynomial.chebyshev * The chebint and chebder functions now check that the specified number integrations or derivations is a non-negative integer. The number 0 is a valid value for both functions. * A degree method has been added to the Chebyshev class. * A trimdeg method has been added to the Chebyshev class. It operates like truncate except that the argument is the desired degree of the result, not the number of coefficients. * Chebyshev.fit now uses None as the default domain for the fit. The default Chebyshev domain can be specified by using [] as the domain value. * Weights can be used in both chebfit and Chebyshev.fit * A linspace method has been added to the Chebyshev class to ease plotting. * The chebmulx function was added. * Added functions for the Chebyshev points of the first and second kind. histogram - After a two years transition period, the old behavior of the histogram function has been phased out, and the new keyword has been removed. correlate - The old behavior of correlate was deprecated in 1.4.0, the new behavior (the usual
[Numpy-discussion] Python3 and intp error (related to ticket 99)
Hi,
I was curious why there was a difference in number of known failures
between Python2.6 and Python3.1 which is associated a test due to ticket 99:
http://projects.scipy.org/numpy/ticket/99
While this ticket was closed, it fails with Python 3.1 as indicated by
the message of the test output is:
'Ticket #99 ... KNOWNFAIL: numpy.intp('0xff', 16) not supported on Py3,
as it does not inherit from Python int'
I do understand that np.intp is integer size of a pointer. But it
appears to be mainly used for access to C programs. The only Python
numpy usage I saw was with the delete and insert function in
'numpy/lib/function_base.py'.
Does this really need to be exposed in Python?
If it does not, then could be removed for Numpy 2?
Otherwise, at the very least np.intp must have the same behavior across
Python versions. As per the ticket, since it is an integer type, should
it have the same properties as a regular integer?
Bruce
$ python3.1
Python 3.1.2 (r312:79147, Mar 24 2010, 10:44:23)
[GCC 4.4.3 20100127 (Red Hat 4.4.3-4)] on linux2
Type help, copyright, credits or license for more information.
import numpy
numpy.__version__
'1.5.0'
numpy.intp('0xff', 16)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: function takes at most 1 argument (2 given)
numpy.intp('0xff')
Traceback (most recent call last):
File stdin, line 1, in module
ValueError: setting an array element with a sequence.
Yet there is no problem with Python 2.6:
$ python
Python 2.6.4 (r264:75706, Jun 4 2010, 18:20:31)
[GCC 4.4.4 20100503 (Red Hat 4.4.4-2)] on linux2
Type help, copyright, credits or license for more information.
import numpy
numpy.__version__
'1.5.0'
numpy.intp('0xff', 16)
255
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Python3 and intp error (related to ticket 99)
Tue, 31 Aug 2010 12:03:55 -0500, Bruce Southey wrote:
[clip]
I do understand that np.intp is integer size of a pointer. But it
appears to be mainly used for access to C programs. The only Python
numpy usage I saw was with the delete and insert function in
'numpy/lib/function_base.py'.
Does this really need to be exposed in Python? If it does not, then
could be removed for Numpy 2?
Yes, it needs to be exposed, since it can be used in data type
definitions.
Otherwise, at the very least np.intp must have the same behavior across
Python versions. As per the ticket, since it is an integer type, should
it have the same properties as a regular integer?
This is not specific to intp, but applies to all Numpy integer types on
Python 3, since they no longer inherit the constructor from the Python
integer.
I note that on Python 2, you cannot do
np.int16('0xff', 16)
either -- it's the same issue. It's also a minor issue, IMHO, as I doubt
many people construct array scalars from strings, and even less do it in
bases other than 10. The fix is to change array scalar __new__, but this
is not completely straightforward to do. Patches are welcome.
--
Pauli Virtanen
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] inversion of large matrices
On 2010-08-30, at 10:36 PM, Charles R Harris wrote: I don't see what the connection with the determinant is. The log determinant will be calculated using the ordinary LU decomposition as that works for more general matrices. I think he means that if he needs both the determinant and to solve the system, it might be more efficient to do the SVD, obtain the determinant from the diagonal values, and obtain the solution by multiplying by U D^-1 V^T? David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Boolean arrays
2010/8/27 Brett Olsen brett.ol...@gmail.com: If there's multiple possible valid values, I've come up with a couple possible methods, but they all seem to be inefficient or kludges: valid = N.array((a, c)) (ar == valid[0]) | (ar == valid[1]) array([ True, False, True, False, False, True, False, True, True], dtype=bool) N.array(map(lambda x: x in valid, ar)) array([ True, False, True, False, False, True, False, True, True], dtype=bool) (ar[..., numpy.newaxis] == valid).T.sum(axis=0).T 0 should also do the job. But it eats up memory. (It employs broadcasting.) Friedrich ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] inversion of large matrices
Hi Melissa, On 30 August 2010 17:42, Melissa Mendonça meliss...@gmail.com wrote: I've been lurking for a while here but never really introduced myself. I'm a mathematician in Brazil working with optimization and numerical analysis and I'm looking into scipy/numpy basically because I want to ditch matlab. Welcome to the list! I hope you will find the switch to numpy and scipy rewarding above and beyond not-being-matlab. Please feel free to ask questions on the list; as you've probably seen, we get lots of questions with simple but non-obvious answers, and a few really tough questions. I'm just curious as to why you say scipy.linalg.solve(), NOT numpy.linalg.solve(). Can you explain the reason for this? I find myself looking for information such as this on the internet but I rarely find real documentation for these things, and I seem to have so many performance issues with python... I'm curious to see what I'm missing here. I agree that the documentation is a little hard to find one's way around sometimes. The numpy documentation project has done a wonderful job of providing detailed documentation for all the functions in numpy, but there's not nearly as much documentation giving a general orientation (which functions are better, what the relationship is with scipy and matplotlib). The scipy documentation project is unfortunately still getting started. What sorts of performance issues have you had? python has some important limitations, but there are often ways to work around them. Sometimes there isn't an efficient way to do things, though, and in those cases we'd like to think about whether numpy/scipy can be improved. (Bulk linear algebra - solving large numbers of small problems - stands out as an example.) Numerical optimization is another - we know the optimizers built into scipy have serious limitations, and welcome ideas on improving them. Thanks, and sorry if I hijacked the thread, No problem, and welcome again to the list. Anne ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] comparison between arrays of strings and numerical types
31/08/10 @ 09:44 (-0700), thus spake Keith Goodman: 2010/8/31 Ernest Adrogué eadro...@gmx.net: Hi, I find this a bit odd: In [18]: np.array(['a','b','c','d']) 'a' Out[18]: array([False, True, True, True], dtype=bool) In [19]: np.array(['a','b','c','d']) 4 Out[19]: True In [20]: np.array(['a','b','c','d']) 4.5 Out[20]: True Is that right? I was expecting an element-wise comparison, but it returns a single truth value. Here's a recent discussion of the issue: http://www.mail-archive.com/numpy-discussion@scipy.org/msg27051.html Ah, I suppose I wasn't paying attention :) There appears to be an open ticket as well: http://projects.scipy.org/numpy/ticket/725 Regards, Ernest ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] inversion of large matrices
On 2010-08-30, at 10:19 PM, Dan Elliott wrote: You don't think this will choke on a large (e.g. 10K x 10K) covariance matrix? That depends. Is it very close to being rank deficient?That would be my main concern. NumPy/LAPACK will have no trouble Cholesky-decomposing a matrix this big, presuming the matrix is well-behaved/full rank. Depending on your hardware it will be slow, but the Cholesky decomposition will be faster (I think usually by about a factor of 2) than other methods that do not assume positive-definiteness. At any rate, inverting the matrix explicitly will take longer and be quite a bit worse in terms of the eventual accuracy of your result. Given what you know about how it computes the log determinant and how the Cholesky decomposition, do you suppose I would be better off using eigen- decomposition to do this since I will also get the determinant from the sum of the logs of the eigenvalues? You could do it this way, though I am not certain of the stability concerns (and I'm in the middle of a move so my copy of Golub and Van Loan is packed up somewhere). I know that the SVD is used in numpy.leastsq, so it can't be that bad. I've never used covariance matrices quite so big that computing the determinant was a significant cost. One thing to note is that you should definitely not be solving the system for every single RHS vector separately. scipy.linalg.solve supports matrix RHSes, and will only do the matrix factorization (be it LU or Cholesky) once, requiring only the O(n^2) forward/backsubstitution to be done repeatedly. This will result in significant savings: In [5]: X = randn(1,2) In [6]: Y = dot(X, X.T) In [7]: timeit scipy.linalg.solve(Y, randn(1), sym_pos=True) 10 loops, best of 3: 78.6 s per loop In [8]: timeit scipy.linalg.solve(Y, randn(1, 50), sym_pos=True) 10 loops, best of 3: 81.6 s per loop David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Github migration?
Hi Chuck (and anyone else interested), I updated the refactoring page on the NumPy developer wiki (seems to be down or I'd paste in the link). It certainly isn't complete, but there are a lot more details about the data structures and memory handling and an outline of some additional topics that needs to be filled in. Regards, Jason On Wed, Aug 25, 2010 at 10:20 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, Aug 25, 2010 at 8:19 AM, Jason McCampbell jmccampb...@enthought.com wrote: Chuck, I will update the wiki page on the Numpy developer site that discusses the refactoring this week. Right now what's there reflects our plans before they met the reality of code. Needless to say, the actual implementation differs in some of the details. Here is a very brief overview of the structure: - The libndarray directory now contains all of the code for the 'core' library. This library is independent of Python and implements most of the array, descriptor, iterator, and ufunc functionality. The goal is that all non-trivial behavior should be in here, but in reality some parts are tied fairly tightly to the CPython interpreter and will take more work to move into the core. - numpy/core/src/multiarray and numpy/core/src/umath now implement just the CPython interface to libndarray. We have preserved both the Python interface and the C API. Ideally each C API function is just a simple wrapper around a call to the core API, though it doesn't always work out that way. However, a large amount of code has been moved out of these modules into the core. - The core is built as a shared library that is independent of any given interface layer. That is, the same shared library/DLL can be used with CPython, IronPython and any other implementation. Each interface is required to pass in a set of callbacks for handling reference counting, object manipulation, and other interface-specific behavior. - The core implements its own reference counting semantics that happen to look very much like CPython's. This was necessary to make the core library independent of the interface layer and preserve performance (ie, limit the number of callbacks). The handshaking between interface and core is a bit complicated but ends up working nicely and efficiently for both reference counted and garbage collected systems. I'll write up the details on the wiki page. - As Travis mentioned we are also working on a .NET back end to Cython. This lets us port the modules such as MTRAND without having to have two separate interfaces, a Cython and a .NET version. Instead, we can modify the existing .pyx file to use the new core API (should improve performance in CPython version slightly). Once done, Cython can generate the .NET and CPython interfaces from the same .pyx file. We have done a fair amount of cleanup on the naming conventions but certainly more needs to be done! I'll write it up for everyone this week but feel free to email me with other questions. Thanks for the summary, it clarifies things a lot. On my cleanup wish list, some of the functions use macros that contain jumps, which is not so nice. I've been intending to scratch that itch for several years now but haven't gotten around to it. I expect such things have a lower priority than getting the basic separation of functionality in place, but just in case... snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] inversion of large matrices
David Warde-Farley dwf at cs.toronto.edu writes: On 2010-08-30, at 10:36 PM, Charles R Harris wrote: I think he means that if he needs both the determinant and to solve the system, it might be more efficient to do the SVD, obtain the determinant from the diagonal values, and obtain the solution by multiplying by U D^-1 V^T? Thank you, that is what I meant. Poorly worded on my part. In particular, I am writing code to invert a very large covariance matrix. I think David has some good information in another post in this thread. - dan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] inversion of large matrices
On Tue, Aug 31, 2010 at 4:52 PM, Dan Elliott danelliotts...@gmail.comwrote: David Warde-Farley dwf at cs.toronto.edu writes: On 2010-08-30, at 10:36 PM, Charles R Harris wrote: I think he means that if he needs both the determinant and to solve the system, it might be more efficient to do the SVD, obtain the determinant from the diagonal values, and obtain the solution by multiplying by U D^-1 V^T? Thank you, that is what I meant. Poorly worded on my part. In particular, I am writing code to invert a very large covariance matrix. I think David has some good information in another post in this thread. Where did the covariance array come from? It may be the case that you can use a much smaller one, for instance in PCA of images. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Where is the dev version of numpydoc?
Hello, I would like to update my numpydoc so it works with sphinx 1.0, but I am not sure where the dev version is; can someone point me in the right direction? John ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
