Re: [Numpy-discussion] Governance model request
On Wed, Sep 23, 2015 at 3:37 PM, Fernando Perez <fperez@gmail.com> wrote: > > Glad to see the other thread proceeding further, let's let this one die as > peacefully as it can... > > This Monty Python sketch seems relevant: https://www.youtube.com/watch?v=grbSQ6O6kbs (hope everyone's in good health and regains better spirits, back to lurking for me) -- _ / \ A* \^ - ,./ _.`\\ / \ / ,--.S\/ \ / `"~,_ \\ __o ? _ \<,_ /:\ --(_)/-(_).../ | \ --...J Paul Ivanov http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Please stop bottom posting!!
Paul Ivanov wrote: ... But I just came across a wonderfully short signature from Rick Moen, and thought I'd pass it along: Cheers, A: Yes. Rick MoenQ: Are you sure? rick@linuxmafia A: Because it reverses the logical flow of conversation. .com McQ! (4x80) Q: Why is top posting frowned upon? This is cute, and makes a good point, but misses the bigger issue: most interesting technical threads are not a series of linear, simple one-liners. Bottom posting is obviously the way to go for that. They are long, with multiple points, and comments on the middle parts, not just the end, and different people all commenting on the same post. It simply does not end up a neat linear conversation anyway. niether pure bottom posting nor pure top posting works well. Full agreed, again, so I'm a little puzzled why you say it misses the point, since I started off that post agreeing with you: Chris Barker - NOAA Federal, on 2013-04-04 09:01, wrote: Which is why I advocate interspersed posting. This. (with heavy snipping). My inclusion of Rick's signature was intended to just be cute, just as you perceived it. best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy discussion - was: Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Ralf Gommers, on 2013-04-09 00:06, wrote: G. website improvement -- thank you, desperately needed. Here's some login credentials and a medal. see https://github.com/numpy/numpy.org/pull/1 Do I really get a medal? ;) best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy discussion - was: Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Matthew Brett, on 2013-04-06 10:39, wrote: Hi, On Sat, Apr 6, 2013 at 10:18 AM, matti picus matti.pi...@gmail.com wrote: as a lurker, may I say that this discussion seems to have become non-productive? Well - the discussion is about two things - as so often the case on numpy. The first is about the particular change. The second is implicit, and keeps coming up, and that is about how change of any sort comes about in numpy. These questions keep coming up. Who decides on change? Is there a veto? Who has it? When do we vote, when do we negotiate? For example, one small part of the discussion was the lack of developers in numpy. For some, stopping these long discussions (somehow) will help recruit developers. The idea is that developers don't like these discussions, and, implicitly, that there is no problem, so the discussion is unnecessary. This is a version of the old 'no man, no problem' solution. For others, these long and unproductive discussions are pointing at a severe problem right at the heart of numpy development, which is that it is very hard to work in a system where it is unclear how decisions get made. See for example Mark Wiebe's complaint at the end here [1]. If this second lot of people are right then we have two options a) stop the discussions, numpy decays and dies from lack of direction b) continue the discussions and hope that it becomes clear that this is indeed a serious problem, and there develops some agreement to fix it. [1] https://github.com/numpy/numpy.org/blob/master/NA-overview.rst I just wanted to chime in that while the document on contributing to numpy [2] is pretty thorough in terms of the technical details expected of a submission - it makes practically no allusions to the social aspects of contributing changes - e.g. the expectations one should have when proposing a change - a loose flowchart or set of possible outcomes. 2. http://docs.scipy.org/doc/numpy/dev/gitwash/development_workflow.html Here's an example of what those expectations might be: 1. your change gets reviewed by a core committer, is deemed acceptable, and gets merged. 2. specific changes are required to the implementation/logic, after which the PR will be deemed acceptable, and get merged 3. the proposal is too broad in scope, or proposes a drastic change that has not been discussed on the list, and should be paused from further action pending the outcome of such a discussion. About a year ago, it seems like something like situation 3 occurred, where folks didn't feel like there was sufficient notice outside of having to pay attention to the Numpy PR queue. In a related note - it should be made clear who the core committers are, at this point. The github organization lists the following eight: charris cournape njsmith pv rgommers rkern seberg teoliphant Is that the core, the whole core, and nothing but the core? -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering
Hi Ralf, Ralf Gommers, on 2013-04-06 10:51, wrote: P.P.S. expect an identical response from me to future proposals that include backwards compatibility breaks of heavily used functions for something that's not a functional enhancement or bug fix. Such proposals are just not OK. but it is a functional enhancement or bug fix - the ambiguity in the affect of order= values in several places only serve to confuse two different ideas into one. -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] www.numpy.org home page
On Mon, Dec 17, 2012 at 12:30 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: Interesting -- I was asked to review the Numpy 1.5 Beginner's Guide, and I did read through the whole thing, and make notes, but never wrote up a full review. One reason is that I found it hard to motivate myself to write what would have been a bad review. This was also my experience. I would go so far as to say that it would be a disservice to our community to link to that book. Our documentation is better. -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] www.numpy.org home page
On Mon, Dec 17, 2012 at 5:50 PM, Paul Ivanov pivanov...@gmail.com wrote: On Mon, Dec 17, 2012 at 12:30 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: Interesting -- I was asked to review the Numpy 1.5 Beginner's Guide, and I did read through the whole thing, and make notes, but never wrote up a full review. One reason is that I found it hard to motivate myself to write what would have been a bad review. This was also my experience. I would go so far as to say that it would be a disservice to our community to link to that book. Our documentation is better. I dug up the skeleton of the review that I had written up until I lost steam and interest in going further, I think it may shed more light on my negative opinion of this book. Packt publishing approached me about doing a review of one of their newest Python books: _NumPy 1.5 Beginner's Guide_ I think it's great that publishers are making it easier for folks to get started in this hot area of computing, though obviously, being a vested member of the Scientific Python community, I'm not exactly unbiased in my opinions on the topic. I received a complementary e-book copy. Here are my thoughts It correctly mentions that NumPy can use a LAPACK implementation if one is available on your system, and also correctly mentions that NumPy provides its own implementation if it can't find one - but neglects to state the important fact that this will be very slow relative to a traditional LAPACK implementation No mention of CPython - since there are so many other flavors of Python out there, these days, and NumPy doesn't work on most of them Mentions that NumPy is open source and free as in beer, but neglects to specifically state its license. Numpy 1.5 is in the book title - but NumPy 2.0.0.dev20100915 listed under Here is a list of software used to develop and test the code examples in the Preface. - the need to register for an account in order to download the sample code is annoying The author of the book uses a 64-bit machine. How do I know that? The first code example provided in the book does not work for the second set of inputs. 20:32@ch1code$ python vectorsum.py 2000 The last 2 elements of the sum [7980015996L, 7992002000L] PythonSum elapsed time in microseconds 4943 Warning: invalid value encountered in power The last 2 elements of the sum [-2143491644 -2143487647] NumPySum elapsed time in microseconds 722 20:32@ch1code$ Indeed the vectorsum example: doesn't work for values 1291 ( int overflow on my 32 bit machine ) python vectorsum.py 1291 20:32@ch1code$ python vectorsum.py 1291 The last 2 elements of the sum [2143362090, 2148353100L] PythonSum elapsed time in microseconds 1771 The last 2 elements of the sum [ 2143362090 -2146614196] NumPySum elapsed time in microseconds 374 So though the answer is attained way faster using numpy, it's wrong in this case! What just happened? section a bit annoying - just space filler without utility - reminiscent of closing curling brackets or line-ending semicolons of those other programming languages :) Ditto with the time for action headings. import numpy as np would have been nice - since that's the convention used in numpy's own doc strings. What does arange(5) do - namespaces are one honking good idea... Re: IPython: The Pylab switch imports all the Scipy, NumPy, and Matplotlib packages. Without this switch, we would have to import every package we need ourselves. - biggest use of ``--pylab`` is to get a separate event loop for plots. It's confusing to have a Numpy book that, in the first chapter, dives into IPython! The book is unfocused in its presentation. A simple mention of %quickref would have sufficed for pointing out features of IPython TypeError for ints and floats is a property of the Python programming language - it's not specific to Numpy. Reshape function should have mentioned that it only changes only the metadata, so reshaping a really large array takes the same amount of time as a small one. More style problems: This selects the first floor Mention of ravel() and flatten() without mention of flat (which is mentioned several pages later). Mention of a.transpose() without a.T - which is mentioned seven pages later. Something I didn't know: The flat attribute is settable. Setting the value of the flat attribute leads to overwriting the values of the whole array (p 46). And then I figured out why I didn't know this (it's slow): In [31]: timeit a.flat=1 10 loops, best of 3: 80.4 ms per loop In [32]: timeit a.fill(1) 100 loops, best of 3: 5.62 ms per loop Unfortunately, the fill() method is *not* mentioned here, it's mentioned 25 pages later. VWA-what? too simplistic explanation to be useful. Mean - cheeky language, but the fact that it's also a method on numpy array is only mentioned in passing four pages later. That you can specify ``axis`` keyword to get means across rows or columns, etc., is also
Re: [Numpy-discussion] import numpy performance
On Mon, Jul 2, 2012 at 2:59 PM, Nathaniel Smith n...@pobox.com wrote:
On Mon, Jul 2, 2012 at 10:06 PM, Robert Kern robert.k...@gmail.com wrote:
On Mon, Jul 2, 2012 at 9:43 PM, Benjamin Root ben.r...@ou.edu wrote:
On Mon, Jul 2, 2012 at 4:34 PM, Nathaniel Smith n...@pobox.com wrote:
I think this ship has sailed, but it'd be worth looking into lazy
importing, where 'numpy.fft' isn't actually imported until someone
starts using it. There are a bunch of libraries that do this, and one
would have to fiddle to get compatibility with all the different
python versions and make sure you're not killing performance (might
have to be in C) but something along the lines of
class _FFTModule(object):
def __getattribute__(self, name):
mod = importlib.import_module(numpy.fft)
_FFTModule.__getattribute__ = mod.__getattribute__
return getattr(mod, name)
fft = _FFTModule()
Not sure how this would impact projects like ipython that does
tab-completion support, but I know that that would drive me nuts in my basic
tab-completion setup I have for my regular python terminal. Of course, in
the grand scheme of things, that really isn't all that important, I don't
think.
We used to do it for scipy. It did interfere with tab completion. It
did drive many people nuts.
Sounds like a bug in your old code, or else the REPLs have gotten
better? I just pasted the above code into both ipython and python
prompts, and typing 'fft.tab' worked fine in both cases. dir(fft)
works first try as well.
(If you try this, don't forget to 'import importlib' first, and note
importlib is 2.7+ only. Obviously importlib is not necessary but it
makes the minimal example less tedious.)
For anyone interested, I worked out a small lazy-loading class that
we use in nitime [1], which does not need importlib and thus works
on python versions before 2.7 and also has a bit of repr pretty
printing.
I wrote about this to Scipy-Dev [2], and in the original nitime
PR [3] tested that it works in python 2.5, 2.6, 2.7, 3.0, 3.1 and
3.2.
Since that time, we've only changed how we deal with the one
known limitation: reloading a lazily-loaded module was a noop in
that PR, but now throws an error (there's one line commented out
if the noop behavior is preferred).
Here's a link to the rendered docs [4], but if you just grab the
LazyImport class from [1], you can do
fft = LazyImport('numpy.fft')
1. https://github.com/nipy/nitime/blob/master/nitime/lazyimports.py
2. http://mail.scipy.org/pipermail/scipy-dev/2011-September/016606.html
3. https://github.com/nipy/nitime/pull/88
4.
http://nipy.sourceforge.net/nitime/api/generated/nitime.lazyimports.html#module-nitime.lazyimports
best,
--
Paul Ivanov
314 address only used for lists, off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Missing data wrap-up and request for comments
On Wed, May 9, 2012 at 3:12 PM, Travis Oliphant tra...@continuum.io wrote: On re-reading, I want to make a couple of things clear: 1) This wrap-up discussion is *only* for what to do for NumPy 1.7 in such a way that we don't tie our hands in the future.I do not believe we can figure out what to do for masked arrays in one short week. What happens beyond NumPy 1.7 should be still discussed and explored.My urgency is entirely about moving forward from where we are in master right now in a direction that we can all accept. The tight timeline is so that we do *something* and move forward. 2) I missed another possible proposal for NumPy 1.7 which is in the write-up that Mark and Nathaniel made: remove the masked array additions entirely possibly moving them to another module like numpy-dtypes. Again, these are only for NumPy 1.7. What happens in any future NumPy and beyond will depend on who comes to the table for both discussion and code-development. I'm glad that this sentence made it into the write-up: A project like numpy requires developers to write code for advancement to occur, and obstacles that impede the writing of code discourage existing developers from contributing more, and potentially scare away developers who are thinking about joining in. I agree, which is why I'm a little surprised after reading the write-up that there's no deference to the alterNEP (admittedly kludgy) implementation? One of the arguments made for the NEP preliminary NA-mask implementation is that has been extensively tested against scipy and other third-party packages, and has been in master in a stable state for a significant amount of time. It is my understanding that the manner in which this implementation found its way into master was a source of concern and contention. To me (and I don't know the level to which this is a technically feasible) that's precisely the reason that BOTH approaches be allowed to make their way into numpy with experimental status. Otherwise, it seems that there is a sort of scaring away of developers - seeing (from the sidelines) how much of a struggle it's been for the alterNEP to find a nurturing environment as an experimental alternative inside numpy. In my reading, the process and consensus threads that have generated so many responses stem precisely from trying to have an atmosphere where everyone is encouraged to join in. The alternatives proposed so far (though I do understand it's only for 1.7) do not suggest an appreciation for the gravity of the fallout from the neglect the alterNEP and the issues which sprang forth from that. Importantly, I find a problem with how personal this document (and discussion) is - I'd much prefer if we talk about technical things by a descriptive name, not the person who thought of it. You'll note how I've been referring to NEP and alterNEP above. One advantage of this is that down the line, if either Mark or Nathaniel change their minds about their current preferred way forward, it doesn't take the wind out of it with something like Even Paul changed his mind and now withdraws his support of Paul's proposal. We should only focus on the technical merits of a given approach, not how many commits have been made by the person proposing them or what else they've done in their life: a good idea has value regardless of who expresses it. In my fantasy world, with both approaches clearly existing in an experimental sandbox inside numpy, folks who feel primary attachments to either NEP or alterNEP would be willing to cross party lines and pitch in towardd making progress in both camps. That's the way we'll find better solutions, by working together, instead of working in opposition. best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Issue Tracking
+1 on migrating issues to GitHub, I'm so glad this discussion is happening. On Sat, May 5, 2012 at 8:28 PM, Charles R Harris charlesr.har...@gmail.com wrote: Uh oh. We are short on developers as is... Which brings up a question, do people need a github account to open an issue? Creating an account on GH is currently required, but it's automated and self-evident how to do that. This little anecdote may say more about me than it does about the Trac instance, but I think it makes a point anyway: I came across a minor numpy or scipy issue last week while running the test suite, which was in a part of the project I don't use, and wanted to quickly report it, (or maybe just add some details to an existing ticket, I don't recall). I went to the GH page first, and saw that only Pull Requests were handled there, so I figured it must still be on the Trac. I went there to try to open a new ticket and then got something like this message: Notice: You are currently not logged in. You may want to do so now. Error: Forbidden TICKET_CREATE privileges are required to perform this operation TracGuide — The Trac User and Administration Guide I tried to login and got an .htaccess login box in the browser - I tried to remember a username password combination that Jarrod set up for me back in 2008? each time I failed, I was then greeted with: Authorization Required This server could not verify that you are authorized to access the document requested. Either you supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required. Apache/2.2.3 (CentOS) Server at projects.scipy.org Port 80 Of course, taking a look now, I should have either been more diligent about finding the forgot your password? link [1] or just created a new username [2], but at the time it seemed like there was no concrete way to proceed forward. With that, the error didn't seem important enough and I decided to get back to the matter at hand - so I gave up. :\ So it really is nice to have everything in one place. When matplotlib had its tickets on SourceForge, I rarely ventured over there to check them, but now that they are on GitHub, everyone with the commit bit gets an email when a new issue is opened, and it makes it a lot easier to pitch in and participate. 1. http://projects.scipy.org/numpy/reset_password 2. http://projects.scipy.org/numpy/register best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: Numpy 1.6.0 beta 1
Ralf Gommers, on 2011-03-23 17:07, wrote: Please test this beta and report any problems on the Numpy mailing list. Hi, I reopened http://projects.scipy.org/numpy/ticket/1768, as 'test_chisquare' still fails for me on a 32 bit machine. Lowering the precision from 14 to 13 decimals makes the test pass, however. Other than that: NumPy version 1.6.0b1 NumPy is installed in /home/pi/.local/lib/python2.6/site-packages/numpy Python version 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3] nose version 0.11.1 --snip-- Ran 3399 tests in 23.052s FAILED (KNOWNFAIL=3, SKIP=4, failures=1) best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 signature.asc Description: Digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to tell if I succeeded to build numpy with amd, umfpack and lapack
Samuel John, on 2011-01-26 15:08, wrote: Hi there! I have successfully built numpy 1.5 on ubuntu lucid (32 for now). I think I got ATLAS/lapack/BLAS support, and if I ldd linalg/lapack_lite.so I see that my libptf77blas.so etc. are successfully linked. :-) However, how to I find out, if (and where) libamd.a and libumfpack.a have been found and (statically) linked. As far as I understand, I they are not present, a fallback in pure python is used, right? Is there a recommended way, I can query against which libs numpy has been built? So I can be sure numpy uses my own compiled versions of libamd, lapack and so forth. Hi Samuel, take a look at numpy.show_config() and scipy.show_config() best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 signature.asc Description: Digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy installation
Waqar Rashid, on 2011-01-02 00:38, wrote: Hi, trying to install numpy on MacOS with python 3.1 Having installation issues. Has anyone managed to install this on the Mac? regards Waqar - you sent this to the IPython-User list, but I think you probably meant to send it to the numpy-discussion list, since your question does not pertain to IPython itself, so I'm forwarding your email there. Also, can you be more specific about what issues you are having? best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 signature.asc Description: Digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Simple shared arrays
Erik Rigtorp, on 2010-12-30 21:30, wrote: Hi, I was trying to parallelize some algorithms and needed a writable array shared between processes. It turned out to be quite simple and gave a nice speed up almost linear in number of cores. Of course you need to know what you are doing to avoid segfaults and such. But I still think something like this should be included with NumPy for power users. This works by inheriting anonymous mmaped memory. Not sure if this works on windows. --snip-- I've successfully used (what I think is) Sturla Molden's shmem_as_ndarray as outline here [1] and here [2] for these purposes. 1. http://groups.google.com/group/comp.lang.python/browse_thread/thread/79fcf022b01b7fc3 2. http://folk.uio.no/sturlamo/python/multiprocessing-tutorial.pdf -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 signature.asc Description: Digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fromfile() for reading text (one more time!)
Christopher Barker, on 2010-01-04 17:05, wrote: Hi folks, I'm taking a look once again at fromfile() for reading text files. I often have the need to read a LOT of numbers form a text file, and it can actually be pretty darn slow do i the normal python way: for line in file: data = map(float, line.strip().split()) or various other versions that are similar. It really does take longer to read the text, split it up, convert to a number, then put that number into a numpy array, than it does to simply read it straight into the array. However, as it stands, fromfile() turn out to be next to useless for anything but whitespace separated text. Full set of ideas here: http://projects.scipy.org/numpy/ticket/909 However, for the moment, I'm digging into the code to address a particular problem -- reading files like this: 123, 65.6, 789 23, 3.2, 34 ... That is comma (or whatever) separated text -- pretty common stuff. The problem with the current code is that you can't read more than one line at time with fromfile: a = np.fromfile(infile, sep=,) will read until it doesn't find a comma, and thus only one line, as there is no comma after each line. As this is a really typical case, I think it should be supported. Just a potshot, but have you tried np.loadtxt? I find it pretty fast. Here is the question: The work of finding the separator is done in: multiarray/ctors.c: fromfile_skip_separator() It looks like it wouldn't be too hard to add some code in there to look for a newline, and consider that a valid separator. However, that would break backward compatibility. So maybe a flag could be passed in, saying you wanted to support newlines. The problem is that flag would have to get passed all the way through to this function (and also for fromstring). I also notice that it supports separators of arbitrary length, which I wonder how useful that is. But it also does odd things with spaces embedded in the separator: , $ # matches all of: ,$# , $# ,$ # Is it worth trying to fix that? In the longer term, it would be really nice to support comments as well, tough that would require more of a re-factoring of the code, I think (though maybe not -- I suppose a call to fromfile_skip_separator() could look for a comment character, then if it found one, skip to where the comment ends -- hmmm. thanks for any feedback, -Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] doctest improvements patch (and possible regressions)
So far, no one has voiced objections, so should I go ahead and check this in? btw, thanks Mike, what about this one: (np.char.lstrip(c, ' ') == np.char.lstrip(c, '')).all() ... # XXX: is this a regression? this line now returns False -pi ... # np.char.lstrip(c,'') does not modify c at all. True best, Paul Ivanov Michael Droettboom, on 2009-12-09 06:04, wrote: Paul Ivanov wrote: I marked up suspicious differences with XXX, since I don't know if they're significant. In particular: - shortening a defchararray by strip does not change it's dtype to a shorter one (apparently it used to?) Yes. The new behavior is to return a string array with the same itemsize as the input array. That's primarily just the result of the new implementation rather than a thought out change, though. Sorry, just commenting on the parts I feel competent in :) But I think this is a great improvement. It would be nice to start doing doctests as a matter of course to keep the docs accurate. Mike ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] doctest improvements patch (and possible regressions)
Hi Numpy-devs, I'm a long time listener, first time caller. I grabbed 1.4.0rc1 and was happy that all the tests passed. But then I tried: import numpy as np np.test(doctests=True) ... Ran 1696 tests in 22.027s FAILED (failures=113, errors=24) I looked at some of the failures, and they looked like trivial typos. So I said to myself: Self, wouldn't it be cool if all the doctests worked? Well, I didn't quite get there spelunking and snorkeling in the source code a few evenings during the past week, but I got close. With the attached patch (which patches the 1.4.0rc1 tarball), I now get: import numpy as np np.test(doctests=True) ... Ran 1696 tests in 20.937s FAILED (failures=33, errors=25) I marked up suspicious differences with XXX, since I don't know if they're significant. In particular: - shortening a defchararray by strip does not change it's dtype to a shorter one (apparently it used to?) - the docstring for seterr says that np.seterr() should reset all errors to defaults, but clearly doesn't do that - there's a regression in recfunctions which may be related to #1299 and may have been fixed - recfunctions.find_duplicates ignoremask flag has no effect. There are a few other things, but they're minor (e.g. I added a note about how missing values are filled with usemask=False in recfunctions.merge_arrays). I think the only code I added was to testing/noseclasses.py. There, if a test fails, I give a few more chances to pass by normalizing the endianness of both desired and actual output, as well as default int size for 32 and 64 bit machines. This is done just using replace() on the strings. Everything else is docstring stuff, so I was hoping to sneak this into 1.4.0, since it would make it that much more polished. Does that sound crazy? best, Paul Ivanov better-doctests.patch.gz Description: GNU Zip compressed data ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Accelerating NumPy computations [Was: GPU Numpy]
Matthew Brett, on 2009-08-21 11:51, wrote: Hi, Indeed. In the future, if OpenCL is the way to go, it may even be helpful to have Numpy using OpenCL directly, as AMD provides an SDK for OpenCL, and with Larrabee approaching, Intel will surely provide one of its own. I was just in a lecture by one of the Intel people about OpenCL: http://parlab.eecs.berkeley.edu/bootcampagenda http://parlab.eecs.berkeley.edu/sites/all/parlab/files/OpenCL_Mattson.pdf He offered no schedule for an Intel OpenCL implementation, but said that they were committed to it. The lectures in general were effective in pointing out what a time-consuming effort it can be moving algorithms into the the parallel world - including GPUs. The lecture just passed cited the example of a CUDA-based BLAS implementation on the GPU that was slower than the CPU version.Making BLAS go faster required a lot of work to find optimal strategies for blocking, transfer between CPU / GPU shared memory / GPU registers, vector sizes and so on - this on a specific NVIDIA architecture. I can imagine Numpy being useful for scripting in this C-and-assembler-centric world, making it easier to write automated testers, or even generate C code. Is anyone out there working on this kind of stuff? I ask only because there seems to be considerable interest here on the Berkeley campus. This is exactly the sort of thing you can do with PyCUDA, which makes it so awesome! http://mathema.tician.de/software/pycuda In particular, see the metaprogramming portion of the docs: http://documen.tician.de/pycuda/metaprog.html The metaprogramming section of the slides and source code from Nicolas Pinto and Andreas Klöckner *excellent* SciPy2009 Tutorials is even more thorough: http://conference.scipy.org/static/wiki/scipy09-pycuda-tut.pdf http://conference.scipy.org/static/wiki/scipy09-pycuda-tut.tar.gz cheers, Paul Ivanov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
