Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
2008/6/23 Alan McIntyre [EMAIL PROTECTED]: Some docstrings have examples of how to use the function that aren't executable code (see numpy.core.defmatrix.bmat for an example) in their current form. Should these examples have the removed from them to avoid them being picked up as doctests? The examples written for the random module warrants the same question. First and foremost, the docstrings are there to illustrate to users how to use the code; second, to serve as tests. Example codes should run, but I'm not sure whether they should always be valid doctests. In the `bmat` example, I would remove the '' like you suggested. Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ndarray methods vs numpy module functions
[ I'm new here and this has the feel of an FAQ but I couldn't find anything at http://www.scipy.org/FAQ . If I should have looked somewhere else a URL would be gratefully received. ] What's the reasoning behind functions like sum() and cumsum() being provided both as module functions (numpy.sum(data, axis=1)) and as object methods (data.sum(axis=1)) but other functions - and I stumbled over diff() - only being provided as module functions? print numpy.__version__ 1.1.0 data = numpy.array([[1,2,3],[4,5,6]]) numpy.sum(data,axis=1) array([ 6, 15]) data.sum(axis=1) array([ 6, 15]) numpy.diff(data,axis=1) array([[1, 1], [1, 1]]) data.diff(axis=1) Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'numpy.ndarray' object has no attribute 'diff' ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] error importing a f2py compiled module.
On Mon, June 23, 2008 10:38 am, Fabrice Silva wrote: Dear all I've tried to run f2py on a fortran file which used to be usable from python some months ago. Following command lines are applied with success (no errors raised) : f2py -m modulename -h tmpo.pyf --overwrite-signature tmpo.f f2py -m modulename -c --f90exec=/usr/bin/f95 tmpo.f First, it is not clear what compiler is f95. If it is gfortran, then use the command f2py -m modulename -c --fcompiler=gnu95 tmpo.f If it is something else, check the output of f2py -c --help-fcompiler and use appropiate --fcompiler switch. Second, I hope you realize that the first command has no effect to the second command. If you have edited the tmpo.pyf file, then use the following second command: f2py tmpo.pyf -c --fcompiler=gnu95 tmpo.f The output of these commands is available here: http://paste.debian.net/7307 When importing in Python with import modulename, I have an ImportError: Traceback (most recent call last): File Solveur.py, line 44, in module import modulename as Modele ImportError: modulename.so: failed to map segment from shared object: Operation not permitted How can that be fixed ? Any suggestion ? I don't have ideas what is causing this import error. Try the instructions above, may be it is due to some compile object conflicts. HTH, Pearu ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Surprising performance tweak in Cython
On Mon, Jun 23, 2008 at 02:03:06AM -0400, Anne Archibald wrote: One way to track down this kind of problem is to look at the C code that's generated. Easier, I admit, with a little familiarity with C, but in any case code working with python variables will be obtrusively strewn with functions like PyFoo; efficient code will be mostly devoid of underscores and the Py prefix. Good point. And cython makes this easy to do with the -a switch that generates anhtml page for that. Thanks for the tip, Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Py_NotImplementedType leak
Leaks of Py_NotImplementedType to the user are regarded as errors by the
Python developers and google turns up several patches fixing such issues. In
numpy we have this problem because we issue the same call for, say,
right_shift as we do for . This leads to things like:
In [1]: int64(1) float32(1)
---
TypeError Traceback (most recent call last)
/home/charris/ipython console in module()
TypeError: unsupported operand type(s) for : 'int' and 'numpy.float32'
In [2]: right_shift(int64(1),float32(1))
Out[2]: NotImplemented
There are several things going on here. First, all the numbers are promoted
to 0-D object arrays, so numpy is using the logic for numpy.object scalars.
Second, the Python interpreter knows how to deal with the operator for
Python ints, which has the left and right slots, but can't deal with the
explicit call to right_shift because it knows nothing about it. Since proper
behavior in this case depends on knowledge of how right_shift is called I
think the thing to do is raise a TypeError{NotImplemented} exception in the
explicit right_shift call and move the NotImplementedType logic up to the
__rrshift__ slot for the numpy array types. This also applies to the other
standard numeric methods.
Thoughts?
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 1:03 AM, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/6/23 Alan McIntyre [EMAIL PROTECTED]: Some docstrings have examples of how to use the function that aren't executable code (see numpy.core.defmatrix.bmat for an example) in their current form. Should these examples have the removed from them to avoid them being picked up as doctests? The examples written for the random module warrants the same question. First and foremost, the docstrings are there to illustrate to users how to use the code; second, to serve as tests. Example codes should run, but I'm not sure whether they should always be valid doctests. In the `bmat` example, I would remove the '' like you suggested. There's also the option of marking them so doctest skips them via #doctest: +SKIP http://docs.python.org/lib/doctest-options.html Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] set_printoptions - floating point format option?
Hi, I need to display some numpy arrays in mantissa+exponent format (e.g. '%.2e' using C syntax). In numpy.set_printoptions(), there is currently only 'precision' option, which does not allow this. What about having an option related to 'precision', named possibly 'float_format', with the following function: :Parameters: format : string Format string for floating point output, has precedence over 'precision' (default '%.8f'). r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 2:02 PM, Fernando Perez [EMAIL PROTECTED] wrote: There's also the option of marking them so doctest skips them via #doctest: +SKIP http://docs.python.org/lib/doctest-options.html For short examples, that seems like a good option, but it seems like you have to have that comment on every line that you want skipped. There are some long examples (like the one in lib/function_base.py:bartlett) that (to me) would look pretty ugly having that comment tacked on to every line. Either way is fine with me in the end, though, so long as it doesn't produce test failures. :) ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 2:37 PM, Pauli Virtanen [EMAIL PROTECTED] wrote: Can you make the convention chosen for the examples (currently only in the doc wiki, not yet in SVN) to work: assuming import numpy as np in examples? This would remove the need for those from numpy import * lines in the examples that I see were added in r5311. Sure, I'll look at that. It seems like every possible option for importing stuff from numpy is used in doctests (sometimes even in the same module), so having them standardized with that implicit import is much better. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
2008/6/23 Fernando Perez [EMAIL PROTECTED]: On Mon, Jun 23, 2008 at 11:17 AM, Alan McIntyre [EMAIL PROTECTED] wrote: On Mon, Jun 23, 2008 at 2:02 PM, Fernando Perez [EMAIL PROTECTED] wrote: There's also the option of marking them so doctest skips them via #doctest: +SKIP http://docs.python.org/lib/doctest-options.html For short examples, that seems like a good option, but it seems like you have to have that comment on every line that you want skipped. There are some long examples (like the one in lib/function_base.py:bartlett) that (to me) would look pretty ugly having that comment tacked on to every line. Ugh. Definitely too ugly if it has to go in every line. From reading the docs I interpreted it as affecting the whole example, which would be far more sensible... Either way is fine with me in the end, though, so long as it doesn't produce test failures. :) Yes, but we also want to make these really easy for users to cleanly paste in with minimal effort. I wonder if a decorator could be applied to those functions so that nose would then skip the doctests: @skip_doctest def foo() Another alternative is to replace +SKIP with something like +IGNORE. That way, the statement is still executed, we just don't care about its outcome. If we skip the line entirely, it often affects the rest of the tests later on. See http://aroberge.blogspot.com/2008/06/monkeypatching-doctest.html for an example implementation. Cheers Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Py_NotImplementedType leak
On Mon, Jun 23, 2008 at 11:23 AM, Charles R Harris
[EMAIL PROTECTED] wrote:
Leaks of Py_NotImplementedType to the user are regarded as errors by the
Python developers and google turns up several patches fixing such issues. In
numpy we have this problem because we issue the same call for, say,
right_shift as we do for . This leads to things like:
In [1]: int64(1) float32(1)
---
TypeError Traceback (most recent call last)
/home/charris/ipython console in module()
TypeError: unsupported operand type(s) for : 'int' and 'numpy.float32'
In [2]: right_shift(int64(1),float32(1))
Out[2]: NotImplemented
There are several things going on here. First, all the numbers are promoted
to 0-D object arrays, so numpy is using the logic for numpy.object scalars.
Second, the Python interpreter knows how to deal with the operator for
Python ints, which has the left and right slots, but can't deal with the
explicit call to right_shift because it knows nothing about it. Since proper
behavior in this case depends on knowledge of how right_shift is called I
think the thing to do is raise a TypeError{NotImplemented} exception in the
explicit right_shift call and move the NotImplementedType logic up to the
__rrshift__ slot for the numpy array types. This also applies to the other
standard numeric methods.
Thoughts?
In particular:
Extension types can now set the type flag Py_TPFLAGS_CHECKTYPES in their
PyTypeObject structure to indicate that they support the new coercion
model. In such extension types, the numeric slot functions can no longer
assume that they'll be passed two arguments of the same type; instead they
may be passed two arguments of differing types, and can then perform their
own internal coercion.
So the NotImplemented return is restricted to numeric slot functions. Given
this, and the fact that leaking NotImplemented to the user is a bug, I'm
going to make the fix unless someone objects.
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 3:21 PM, Stéfan van der Walt [EMAIL PROTECTED] wrote: Another alternative is to replace +SKIP with something like +IGNORE. That way, the statement is still executed, we just don't care about its outcome. If we skip the line entirely, it often affects the rest of the tests later on. Ugh. That just seems like a lot of unreadable ugliness to me. If this comment magic is the only way to make that stuff execute properly under doctest, I think I'd rather just skip it in favor of clean, uncluttered, non-doctestable code samples in the docstrings. If the code that's currently in docstrings needs to be retained as test code, I'll gladly take the time to put it into a test_ module where it doesn't get in the way of documentation. I'll defer to the consensus, though. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
2008/6/23 Alan McIntyre [EMAIL PROTECTED]: On Mon, Jun 23, 2008 at 3:21 PM, Stéfan van der Walt [EMAIL PROTECTED] wrote: Another alternative is to replace +SKIP with something like +IGNORE. That way, the statement is still executed, we just don't care about its outcome. If we skip the line entirely, it often affects the rest of the tests later on. Ugh. That just seems like a lot of unreadable ugliness to me. If this comment magic is the only way to make that stuff execute properly under doctest, I think I'd rather just skip it in favor of clean, uncluttered, non-doctestable code samples in the docstrings. If the code that's currently in docstrings needs to be retained as test code, I'll gladly take the time to put it into a test_ module where it doesn't get in the way of documentation. I'll defer to the consensus, though. I think doctests are valuable: it's very hard for the documentation to get out of sync with the code, and it makes it very easy to write tests, particularly in light of the wiki documentation framework. But I think encrusting examples with weird comments will be a pain for documentors and off-putting to users. Perhaps doctests can be positively marked, in some relatively unobtrusive way? Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
Mon, 23 Jun 2008 10:03:28 +0200, Stéfan van der Walt wrote: 2008/6/23 Alan McIntyre [EMAIL PROTECTED]: Some docstrings have examples of how to use the function that aren't executable code (see numpy.core.defmatrix.bmat for an example) in their current form. Should these examples have the removed from them to avoid them being picked up as doctests? The examples written for the random module warrants the same question. First and foremost, the docstrings are there to illustrate to users how to use the code; second, to serve as tests. Example codes should run, but I'm not sure whether they should always be valid doctests. In the `bmat` example, I would remove the '' like you suggested. Schematic code (such as that currently in numpy.bmat) that doesn't run probably shouldn't be written with , and for it the ReST block quote syntax is also looks OK. But I'm personally not in favor of a distinction between a doctest and a code sample, as the difference is not of interest to the main target audience who reads the docstrings (or the reference documentation generated based on them). As I see it, Numpy has a test architecture that is separate from doctests, so that most of the bonus doctests gives us is ensuring that all of our examples run without errors and produce expected results. It's a bit unfortunate though that the doctest directives are as obtrusive as they are and only apply to a single line. One problem that I see now is quite annoying in the sample codes using matplotlib: matplotlib functions tend to return some objects whose repr contains a memory address, which causes the code to fit badly in a doctest. This can be worked around (ELLIPSIS, assigning to a variable), but I don't see a clean way. (I'm not so worried here about plot windows popping up as they can be worked around by monkey-patching matplotlib.show and choosing a non-graphical backend.) Another point related to numpy are blank lines often appearing in array printouts (the text BLANKLINE is not a pretty sight in documentation). Also, NORMALIZE_WHITESPACE is useful for reducing the whitespace in array printout. -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On 23 Jun 2008, at 12:37 PM, Alan McIntyre wrote:
Ugh. That just seems like a lot of unreadable ugliness to me. If
this comment magic is the only way to make that stuff execute
properly
under doctest, I think I'd rather just skip it in favor of clean,
uncluttered, non-doctestable code samples in the docstrings.
Another perspective: doctests ensure that documentation stays up to
date (if the behaviour or interface changes, then tests will fail
indicating that the documentation also needs to be updated.)
Thus, one can argue that all examples should also be doctests. This
generally makes things a little more ugly, but much less ambiguous.
...
Examples:
-
If A, B, C, and D are appropriately shaped 2-d arrays, then one can
produce
[ A B ]
[ C D ]
using any of these methods:
A, B, C, D = [[1,1]], [[2,2]], [[3,3]], [[4,4]]
np.bmat('A, B; C, D') # From a string
matrix([[ 1, 1, 2, 2],
[ 3, 3, 4, 4]])
np.bmat([[A,B],[C,D]]) # From a nested sequence
matrix([[ 1, 1, 2, 2],
[ 3, 3, 4, 4]])
np.bmat(np.r_[np.c_[A,B],np.c_[C,D]]) # From an array
matrix([[ 1, 1, 2, 2],
[ 3, 3, 4, 4]])
Michael.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
Mon, 23 Jun 2008 15:53:55 -0400, Anne Archibald wrote: 2008/6/23 Alan McIntyre [EMAIL PROTECTED]: On Mon, Jun 23, 2008 at 3:21 PM, Stéfan van der Walt [EMAIL PROTECTED] wrote: Another alternative is to replace +SKIP with something like +IGNORE. That way, the statement is still executed, we just don't care about its outcome. If we skip the line entirely, it often affects the rest of the tests later on. Ugh. That just seems like a lot of unreadable ugliness to me. If this comment magic is the only way to make that stuff execute properly under doctest, I think I'd rather just skip it in favor of clean, uncluttered, non-doctestable code samples in the docstrings. If the code that's currently in docstrings needs to be retained as test code, I'll gladly take the time to put it into a test_ module where it doesn't get in the way of documentation. I'll defer to the consensus, though. I think doctests are valuable: it's very hard for the documentation to get out of sync with the code, and it makes it very easy to write tests, particularly in light of the wiki documentation framework. But I think encrusting examples with weird comments will be a pain for documentors and off-putting to users. Perhaps doctests can be positively marked, in some relatively unobtrusive way? I also think being able to test that the examples in docstrings run correctly could be valuable, but I'm not sure if it makes sense to have this enabled in the default test set. Another idea (in addition to whitelisting): how easy would it be to subclass doctest.DocTestParser so that it would eg. automatically +IGNORE any doctest lines containing plt.? -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 3:57 PM, Pauli Virtanen [EMAIL PROTECTED] wrote:
Schematic code (such as that currently in numpy.bmat) that doesn't run
probably shouldn't be written with , and for it the ReST block quote
syntax is also looks OK.
But I'm personally not in favor of a distinction between a doctest and
a code sample, as the difference is not of interest to the main target
audience who reads the docstrings (or the reference documentation
generated based on them). As I see it, Numpy has a test architecture that
is separate from doctests, so that most of the bonus doctests gives us is
ensuring that all of our examples run without errors and produce expected
results.
I agree with you, Anne and Michael that ensuring that the
documentation examples run is important. The more I think about it,
the more I'd rather have examples that are a bit verbose. In the
particular example of bmat, as a new user, I'd really honestly rather
see those three cases fully coded:
A=nd.arange(1,5).reshape(2,2)
B= etc.
F=bmat('A,B;C,D')
F
matrix([[1,2,5,6],
etc.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 4:07 PM, Pauli Virtanen [EMAIL PROTECTED] wrote: Another idea (in addition to whitelisting): how easy would it be to subclass doctest.DocTestParser so that it would eg. automatically +IGNORE any doctest lines containing plt.? I'll play around with that and see how hard it is to just ignore the ugly bits that currently require all that per-line directive stuff. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
2008/6/23 Michael McNeil Forbes [EMAIL PROTECTED]: On 23 Jun 2008, at 12:37 PM, Alan McIntyre wrote: Ugh. That just seems like a lot of unreadable ugliness to me. If this comment magic is the only way to make that stuff execute properly under doctest, I think I'd rather just skip it in favor of clean, uncluttered, non-doctestable code samples in the docstrings. Another perspective: doctests ensure that documentation stays up to date (if the behaviour or interface changes, then tests will fail indicating that the documentation also needs to be updated.) Thus, one can argue that all examples should also be doctests. This generally makes things a little more ugly, but much less ambiguous. This is a bit awkward. How do you give an example for a random-number generator? Even if you are willing to include a seed in each statement, misleading users into thinking it's necessary, the value returned for a given seed is not necessarily part of the interface a random-number generator agrees to support. I do agree that as many examples as possible should be doctests, but I don't think we should restrict the examples we are allowed to give to only those that can be made to serve as doctests. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Sparse Matrices in Numpy -- (with eigenvalue algorithms if possible)
I'm wondering if there is a currently-available Sparse-Matrix package for numpy? If so, how do I get it?And, if there is a good sparse-matrix package, does it include an eigenvalue-computation algorithm? How would a sparse-matrix package interact with something like numpy.linalg.eig, or for that matter any of the other numpy modules? Thanks, Dan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On 23 Jun 2008, at 1:28 PM, Anne Archibald wrote: 2008/6/23 Michael McNeil Forbes [EMAIL PROTECTED]: Thus, one can argue that all examples should also be doctests. This generally makes things a little more ugly, but much less ambiguous. This is a bit awkward. How do you give an example for a random-number generator? Even if you are willing to include a seed in each statement, misleading users into thinking it's necessary, the value returned for a given seed is not necessarily part of the interface a random-number generator agrees to support. I agree that this can be awkward sometimes, and should certainly not be policy, but one can usually get around this. Instead of printing the result, you can use it, or demonstrate porperties: random_array = np.random.rand(3,4) random_array.shape (3,4) random_array.max() 1 True random_array.min() 0 True etc. Michael. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 15:44, Michael McNeil Forbes [EMAIL PROTECTED] wrote: On 23 Jun 2008, at 1:28 PM, Anne Archibald wrote: 2008/6/23 Michael McNeil Forbes [EMAIL PROTECTED]: Thus, one can argue that all examples should also be doctests. This generally makes things a little more ugly, but much less ambiguous. This is a bit awkward. How do you give an example for a random-number generator? Even if you are willing to include a seed in each statement, misleading users into thinking it's necessary, the value returned for a given seed is not necessarily part of the interface a random-number generator agrees to support. I agree that this can be awkward sometimes, and should certainly not be policy, but one can usually get around this. Instead of printing the result, you can use it, or demonstrate porperties: random_array = np.random.rand(3,4) random_array.shape (3,4) random_array.max() 1 True random_array.min() 0 True Yes, this makes it doctestable, but you've destroyed the exampleness. It should be policy *not* to do this. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 4:51 PM, Robert Kern [EMAIL PROTECTED] wrote: I agree that this can be awkward sometimes, and should certainly not be policy, but one can usually get around this. Instead of printing the result, you can use it, or demonstrate porperties: random_array = np.random.rand(3,4) random_array.shape (3,4) random_array.max() 1 True random_array.min() 0 True Yes, this makes it doctestable, but you've destroyed the exampleness. It should be policy *not* to do this. So it seems we have: 1. Example code that is doctestable 2. Example code that probably can't ever be doctestable (random number stuff, etc.), but is still executable 3. Schematic examples that aren't executable Personally, I'm in favor of filling out examples of type #3 to make them at least #2, but maybe that's not always practical. I don't think #3 should ever have prompts, so it shouldn't ever be picked up by doctest. I suppose I could go for a decorator option to flag #2. If we execute them, but not look at the results, then at least we find out about examples that are broken enough to raise exceptions. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 16:51, Michael McNeil Forbes [EMAIL PROTECTED] wrote: On Mon, Jun 23, 2008 at 4:51 PM, Robert Kern [EMAIL PROTECTED] wrote: random_array = np.random.rand(3,4) random_array.shape (3,4) random_array.max() 1 True random_array.min() 0 True Yes, this makes it doctestable, but you've destroyed the exampleness. It should be policy *not* to do this. Well perhaps... but do you think that rand(d0, d1, ..., dn) - random values is more exampley than r = np.random.rand(3,2,4) r.shape (3,2,4) ? No. It wasn't an example. It was a specification of the call signature because it is in an extension module, so the call signature is not available like it is for pure Python functions. Thus, it needs to be given in the docstring. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ndarray methods vs numpy module functions
On Mon, Jun 23, 2008 at 10:31 AM, Bob Dowling [EMAIL PROTECTED] wrote: [ I'm new here and this has the feel of an FAQ but I couldn't find anything at http://www.scipy.org/FAQ . If I should have looked somewhere else a URL would be gratefully received. ] What's the reasoning behind functions like sum() and cumsum() being provided both as module functions (numpy.sum(data, axis=1)) and as object methods (data.sum(axis=1)) but other functions - and I stumbled over diff() - only being provided as module functions? Hi Bob, this is a very good question. I think the answers are a) historical reasons AND, more importantly, differing personal preferences b) I would file the missing data.diff() as a bug. There are many inconsistencies left in such a big project like numpy. And filing bugs might be the best way of keeping track of them and getting them fixes eventually... (( a much more dangerous example is numpy.resize and data.resize, which do (slightly) different things !!)) Others, please correct my . Welcome on the list. - Sebastian Haase ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ndarray methods vs numpy module functions
On Mon, Jun 23, 2008 at 18:10, Sebastian Haase [EMAIL PROTECTED] wrote: On Mon, Jun 23, 2008 at 10:31 AM, Bob Dowling [EMAIL PROTECTED] wrote: [ I'm new here and this has the feel of an FAQ but I couldn't find anything at http://www.scipy.org/FAQ . If I should have looked somewhere else a URL would be gratefully received. ] What's the reasoning behind functions like sum() and cumsum() being provided both as module functions (numpy.sum(data, axis=1)) and as object methods (data.sum(axis=1)) but other functions - and I stumbled over diff() - only being provided as module functions? Hi Bob, this is a very good question. I think the answers are a) historical reasons AND, more importantly, differing personal preferences b) I would file the missing data.diff() as a bug. It's not. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
2008/6/23 Michael McNeil Forbes [EMAIL PROTECTED]: One can usually do #3 - #1 or #2 by just leave bare assignments without printing a result (the user can always execute them and look at the result if they want). r = np.random.rand(3,2,4) which is cleaner than adding any flags... Purposefully reducing the clarity of an example to satisfy some tool is not an option. We might be able to work around this specific case, but there will be others. It should be fairly easy to execute the example code, just to make sure it runs. We can always work out a scheme to test its validity later. One route is to use the same docstring scraper we use for the reference guide, to extract all tests. We can then choose a markup which identifies tests with unpredictable results, and refrain from executing them. In some instances, we can even infer which tests to ignore, e.g. the ' plt.' example Pauli mentioned. Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
2008/6/24 Stéfan van der Walt [EMAIL PROTECTED]: It should be fairly easy to execute the example code, just to make sure it runs. We can always work out a scheme to test its validity later. Mike Hansen just explained to me that the Sage doctest system sets the random seed before executing each test. If we address a) Random variables b) Plotting representations and c) Endianness we're probably halfway there. Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Sparse Matrices in Numpy -- (with eigenvalue algorithms if possible)
Hi Dan 2008/6/23 Dan Yamins [EMAIL PROTECTED]: I'm wondering if there is a currently-available Sparse-Matrix package for numpy? If so, how do I get it?And, if there is a good sparse-matrix package, does it include an eigenvalue-computation algorithm? How would a sparse-matrix package interact with something like numpy.linalg.eig, or for that matter any of the other numpy modules? SciPy contains all this functionality as `scipy.sparse`. The NumPy algorithms only work on dense matrices, but SciPy provides special linear algebra routines. Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
Stéfan van der Walt wrote: 2008/6/24 Stéfan van der Walt [EMAIL PROTECTED]: It should be fairly easy to execute the example code, just to make sure it runs. We can always work out a scheme to test its validity later. Hi, Mike Hansen just explained to me that the Sage doctest system sets the random seed before executing each test. If we address a) Random variables we have some small extensions to the doctesting framework that allow us to mark doctests as #random so that the result it not checked. Carl Witty wrote some code that makes the random number generator in a lot of the Sage components behave consistently on all supported platforms. b) Plotting representations and c) Endianness Yeah, the Sage test suite seems to catch at least one of those in every release cycle. Another thing we just implemented is a jar of pickles that lets us verify that there is no cross platform issues (32 vs. 64 bits and big vs. little endian) as well as no problems with loading pickles from previous releases. we're probably halfway there. Regards Stéfan Cheers, Michael ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
Fernando Perez wrote: On Mon, Jun 23, 2008 at 4:58 PM, Michael Abshoff [EMAIL PROTECTED] wrote: Hi Fernando, a) Random variables we have some small extensions to the doctesting framework that allow us to mark doctests as #random so that the result it not checked. Carl Witty wrote some code that makes the random number generator in a lot of the Sage components behave consistently on all supported platforms. Care to share? (BSD, we can't even look at the Sage code). I am not the author, so I need to find out who wrote the code, but I am sure it can be made BSD. We are also working on doctest+timeit to hunt for performance regressions, but that one is not ready for prime time yet. Cheers, f Cheers, Michael ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ndarray methods vs numpy module functions
Robert Kern wrote: On Mon, Jun 23, 2008 at 18:10, Sebastian Haase [EMAIL PROTECTED] wrote: On Mon, Jun 23, 2008 at 10:31 AM, Bob Dowling [EMAIL PROTECTED] wrote: [ I'm new here and this has the feel of an FAQ but I couldn't find anything at http://www.scipy.org/FAQ . If I should have looked somewhere else a URL would be gratefully received. ] What's the reasoning behind functions like sum() and cumsum() being provided both as module functions (numpy.sum(data, axis=1)) and as object methods (data.sum(axis=1)) but other functions - and I stumbled over diff() - only being provided as module functions? Hi Bob, this is a very good question. I think the answers are a) historical reasons AND, more importantly, differing personal preferences b) I would file the missing data.diff() as a bug. It's not. Care to elaborate? -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 5:26 PM, Michael Abshoff [EMAIL PROTECTED] wrote: I am not the author, so I need to find out who wrote the code, but I am sure it can be made BSD. We are also working on doctest+timeit to hunt for performance regressions, but that one is not ready for prime time yet. Great, thanks. Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ndarray methods vs numpy module functions
On Mon, Jun 23, 2008 at 19:35, Ryan May [EMAIL PROTECTED] wrote: Robert Kern wrote: On Mon, Jun 23, 2008 at 18:10, Sebastian Haase [EMAIL PROTECTED] wrote: On Mon, Jun 23, 2008 at 10:31 AM, Bob Dowling [EMAIL PROTECTED] wrote: [ I'm new here and this has the feel of an FAQ but I couldn't find anything at http://www.scipy.org/FAQ . If I should have looked somewhere else a URL would be gratefully received. ] What's the reasoning behind functions like sum() and cumsum() being provided both as module functions (numpy.sum(data, axis=1)) and as object methods (data.sum(axis=1)) but other functions - and I stumbled over diff() - only being provided as module functions? Hi Bob, this is a very good question. I think the answers are a) historical reasons AND, more importantly, differing personal preferences b) I would file the missing data.diff() as a bug. It's not. Care to elaborate? There is not supposed to be a one-to-one correspondence between the functions in numpy and the methods on an ndarray. There is some duplication between the two, but that is not a reason to make more duplication. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 5:58 PM, Michael Abshoff [EMAIL PROTECTED] wrote: Stéfan van der Walt wrote: 2008/6/24 Stéfan van der Walt [EMAIL PROTECTED]: It should be fairly easy to execute the example code, just to make sure it runs. We can always work out a scheme to test its validity later. Hi, Mike Hansen just explained to me that the Sage doctest system sets the random seed before executing each test. If we address a) Random variables we have some small extensions to the doctesting framework that allow us to mark doctests as #random so that the result it not checked. Carl Witty wrote some code that makes the random number generator in a lot of the Sage components behave consistently on all supported platforms. But there is more than one possible random number generator. If you do that you are tied into one kind of generator and one kind of initialization implementation. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
Charles R Harris wrote: On Mon, Jun 23, 2008 at 5:58 PM, Michael Abshoff [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Stéfan van der Walt wrote: 2008/6/24 Stéfan van der Walt [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: It should be fairly easy to execute the example code, just to make sure it runs. We can always work out a scheme to test its validity later. Hi, Mike Hansen just explained to me that the Sage doctest system sets the random seed before executing each test. If we address a) Random variables we have some small extensions to the doctesting framework that allow us to mark doctests as #random so that the result it not checked. Carl Witty wrote some code that makes the random number generator in a lot of the Sage components behave consistently on all supported platforms. Hi, But there is more than one possible random number generator. If you do that you are tied into one kind of generator and one kind of initialization implementation. Chuck Correct, but so far Carl has hooked into six out of the many random number generators in the various components of Sage. This way we can set a global seed and also more easily reproduce issues with algorithms where randomness plays a role without being forced to be on the same platform. There are still doctests in Sage where the randomness comes from sources not in randgen (Carl's code), but sooner or later we will get around to all of them. Cheers, Michael ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
2008/6/23 Stéfan van der Walt [EMAIL PROTECTED]: 2008/6/24 Stéfan van der Walt [EMAIL PROTECTED]: It should be fairly easy to execute the example code, just to make sure it runs. We can always work out a scheme to test its validity later. Mike Hansen just explained to me that the Sage doctest system sets the random seed before executing each test. If we address a) Random variables b) Plotting representations and c) Endianness we're probably halfway there. I agree (though I have reservations about how they are to be addressed). But in the current setting, halfway there is still a problem - it seems to me we need, now and later, a way to deal with generic examples that are not doctests. There may not be many of them, and most may be dealt with by falling into categories a, b, and c above, but it is important that we not make it difficult to write new examples even if they can't readily be made into doctests. In particular, we don't want some documentor saying well, I'd like to write an example, but I don't remember the arcane syntax to prevent this failing a doctest, so I'm not going to bother. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
2008/6/23 Michael Abshoff [EMAIL PROTECTED]: Charles R Harris wrote: On Mon, Jun 23, 2008 at 5:58 PM, Michael Abshoff [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Stéfan van der Walt wrote: 2008/6/24 Stéfan van der Walt [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: It should be fairly easy to execute the example code, just to make sure it runs. We can always work out a scheme to test its validity later. Hi, Mike Hansen just explained to me that the Sage doctest system sets the random seed before executing each test. If we address a) Random variables we have some small extensions to the doctesting framework that allow us to mark doctests as #random so that the result it not checked. Carl Witty wrote some code that makes the random number generator in a lot of the Sage components behave consistently on all supported platforms. Hi, But there is more than one possible random number generator. If you do that you are tied into one kind of generator and one kind of initialization implementation. Chuck Correct, but so far Carl has hooked into six out of the many random number generators in the various components of Sage. This way we can set a global seed and also more easily reproduce issues with algorithms where randomness plays a role without being forced to be on the same platform. There are still doctests in Sage where the randomness comes from sources not in randgen (Carl's code), but sooner or later we will get around to all of them. Doesn't this mean you can't change your implementation of random number generators (for example choosing a different implementation of generation of normally-distributed random numbers, or replacing the Mersenne Twister) without causing countless doctests to fail meaninglessly? Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Code samples in docstrings mistaken as doctests
On Mon, Jun 23, 2008 at 22:53, Anne Archibald [EMAIL PROTECTED] wrote: 2008/6/23 Michael Abshoff [EMAIL PROTECTED]: Correct, but so far Carl has hooked into six out of the many random number generators in the various components of Sage. This way we can set a global seed and also more easily reproduce issues with algorithms where randomness plays a role without being forced to be on the same platform. There are still doctests in Sage where the randomness comes from sources not in randgen (Carl's code), but sooner or later we will get around to all of them. Doesn't this mean you can't change your implementation of random number generators (for example choosing a different implementation of generation of normally-distributed random numbers, or replacing the Mersenne Twister) without causing countless doctests to fail meaninglessly? It's not that bad. After you've verified that your new code works, you regenerate the examples. You check in both at the same time. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Sparse Matrices in Numpy -- (with eigenvalue algorithms if possible)
On Tue, Jun 24, 2008 at 4:23 AM, Dan Yamins [EMAIL PROTECTED] wrote: 3) Finally: (and this is slightly off topic) -- I've just installed SciPy for the first time. In the readme.txt for the download it mentioned something about having LAPACK and ATLAS installed. However, on the scipy install page (for OSX, http://www.scipy.org/Installing_SciPy/Mac_OS_X), it doesn't mention anything about LAPACK and ATLAS -- and the instructions there that I followed worked without any need for those packages.Do I need them? Or are they only necessary for making certain routines faster? On OS X it is usual to use the optimised BLAS provided by the Accelerate framework (at least that's what it used to be called I think). In any case - you don't need them because Apple provides an optimised blas (which is used as an alternative to lapack and atlas)... You could use lapack/atlas if you wanted but installation is probably simpler following the instructions on the wiki to use the apple one... Robin ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
