Robert Kern wrote:
It's still pretty bad, though. I do recommend running Disk Repair like Bill
did.
I did that, and it found and did nothing -- I suspect it ran when I
re-booted -- it did take a while to reboot.
However, this is pretty consistently what I'm getting now:
$ time python -c
Robert Kern wrote:
It isn't. The problem is on Chris's file system.
Thanks for all your help, Robert. Interestingly, I haven't noticed any
problems anywhere else, but who knows?
I guess this is what Linux Torvalds meant when he said that OS-X's file
system was brain dead
Whatever is wrong
On Mon, Aug 4, 2008 at 14:24, Christopher Barker [EMAIL PROTECTED] wrote:
Robert Kern wrote:
It isn't. The problem is on Chris's file system.
Thanks for all your help, Robert. Interestingly, I haven't noticed any
problems anywhere else, but who knows?
I guess this is what Linux Torvalds
OK,
So I'm an idiot. After reading this, I thought I haven't rebooted for a
while. It turns out it's been 35 days. I think I've been having slow
startup for a longer than that, but none the less, I re-booted (which
took a long time), and presto:
$ time python -c import numpy
real0m0.686s
On Mon, Aug 4, 2008 at 18:01, Christopher Barker [EMAIL PROTECTED] wrote:
OK,
So I'm an idiot. After reading this, I thought I haven't rebooted for a
while. It turns out it's been 35 days. I think I've been having slow
startup for a longer than that, but none the less, I re-booted (which
On Sat, Aug 2, 2008 at 00:06, David Cournapeau
[EMAIL PROTECTED] wrote:
Christopher Barker wrote:
OK, I just installed wxPython, and whoa!
time python -c import numpy
real0m2.793s
user0m0.294s
sys 0m2.494s
so it's taking almost two seconds more to import numpy, now that
Robert Kern wrote:
It isn't. The problem is on Chris's file system. Whatever is wrong
with his file system (Bill Spotz's identical problem suggests too many
temporary but unused inodes) increases the traversal of the file
system.
Ah, I did not think it could indeed affect the whole fs. This
I've got a proof of concept that take the time on my machine to
import numpy from 0.21 seconds down to 0.08 seconds. Doing that
required some somewhat awkward things, like deferring all 'import re'
statements. I don't think that's stable in the long run because
people will blithely
On Fri, Aug 01, 2008 at 09:18:48AM -0700, Christopher Barker wrote:
What does python -c import sys; print sys.path say ?
A lot! 41 entries, and lot's of eggs -- are eggs an issue? I'm also
wondering how the order is determined -- if it looked in site-packages
first, it would find numpy a
On Fri, Aug 1, 2008 at 11:53, Gael Varoquaux
[EMAIL PROTECTED] wrote:
On Fri, Aug 01, 2008 at 09:18:48AM -0700, Christopher Barker wrote:
What does python -c import sys; print sys.path say ?
A lot! 41 entries, and lot's of eggs -- are eggs an issue? I'm also
wondering how the order is
On Thu, Jul 31, 2008 at 10:02 PM, Robert Kern [EMAIL PROTECTED] wrote:
On Thu, Jul 31, 2008 at 05:43, Andrew Dalke [EMAIL PROTECTED] wrote:
On Jul 31, 2008, at 12:03 PM, Robert Kern wrote:
But you still can't remove them since they are being used inside
numerictypes. That's why I labeled them
David Cournapeau wrote:
IOW, I don't think the problem is the numbers themselves. It has to be
something else. A simple profiling like
python -m cProfile -o foo.stats foo.py
and then:
python -c import pstats; p = pstats.Stats(foo.stats);
p.sort_stats('cumulative').print_stats(50)
OK, see
Robert Kern wrote:
File/Save As..., pick a file name. When asked about whether to embed
source files or strip them out, choose Strip. Then email the resulting
.mshark file to me.
I've done that, and sent it to you directly -- it's too big to put in
the mailing list.
It looks like your
On Sat, Aug 2, 2008 at 5:33 AM, Ondrej Certik [EMAIL PROTECTED] wrote:
But I am still unhappy about it, I'd like if the package could import
much faster, because it adds up, when you need to import 7 packages
like that, it's suddenly 1s and that's just too much.
Too much for what ? We need
On Jul 31, 2008, at 3:53 AM, David Cournapeau wrote:
You are supposed to run the tests on an installed numpy, not in the
sources:
import numpy
numpy.test(verbose = 10)
Doesn't that make things more cumbersome to test? That is, if I were
to make a change I would need to:
- python
Hi All,
I've been reading this discussion with interest.
I would just to highlight an alternate use of numpy to interactive use. We
have a cluster of machines which process tasks on an individual basis where
a master tasks may spawn 600 slave tasks to be processed. These tasks are
spread across
On Thu, Jul 31, 2008 at 7:31 AM, Hanni Ali [EMAIL PROTECTED] wrote:
I would just to highlight an alternate use of numpy to interactive use. We
have a cluster of machines which process tasks on an individual basis where
a master tasks may spawn 600 slave tasks to be processed. These tasks are
Nathan Bell wrote:
There are other components of NumPy/SciPy that are more worthy of
optimization. Given that programmer time is a scarce resource, it's
more sensible to direct our efforts towards making the other 98.5% of
the computation faster.
To be fair, when I took a look at the
On Thu, Jul 31, 2008 at 5:36 AM, Andrew Dalke [EMAIL PROTECTED] wrote:
The user base for numpy might be .. 10,000 people? 100,000 people?
Let's go with the latter, and assume that with command-line scripts,
CGI scripts, and the other programs that people write in order to
help do research
On Thu, Jul 31, 2008 at 03:41:15PM +0900, David Cournapeau wrote:
Yes. Nothing that an easy make file cannot solve, nonetheless (I am sure
I am not the only one with a makefile/script which automates the above,
to test a new svn updated numpy in one command).
That's why distutils have a test
On Thu, Jul 31, 2008 at 12:43:17PM +0200, Andrew Dalke wrote:
Startup performance has not been a numpy concern. It a concern for
me, and it has been (for other packages) a concern for some of my
clients.
I am curious, if startup performance is a problem, I guess it is because
you are
On Thu, Jul 31, 2008 at 2:12 AM, Andrew Dalke [EMAIL PROTECTED] wrote:
Hmm. And it looks like testing/nosetester.py (which implements the
'test' function above) is meant to make it easier to run nose, except
my feeling is the extra level of wrapping makes things more
complicated. The nosetest
Gael Varoquaux wrote:
That's why distutils have a test target. You can do python setup.py
test, and if you have setup you setup.py properly it should work
(obviously it is easy to make this statement, and harder to get the thing
working).
I have already seen some discussion about
On Thu, Jul 31, 2008 at 11:05:33PM +0900, David Cournapeau wrote:
Gael Varoquaux wrote:
That's why distutils have a test target. You can do python setup.py
test, and if you have setup you setup.py properly it should work
(obviously it is easy to make this statement, and harder to get the
Gael Varoquaux wrote:
Obviously, the build part has to be well-tuned for the machinery to work,
but there is a lot of value here.
Ah yes, setuptools does have this. But this is specific to setuptools,
bare distutils does not have this test command, right ?
cheers,
David
On Thu, Jul 31, 2008 at 10:14 AM, Gael Varoquaux
[EMAIL PROTECTED] wrote:
On Thu, Jul 31, 2008 at 12:43:17PM +0200, Andrew Dalke wrote:
Startup performance has not been a numpy concern. It a concern for
me, and it has been (for other packages) a concern for some of my
clients.
I am
On Thu, Jul 31, 2008 at 11:16:12PM +0900, David Cournapeau wrote:
Gael Varoquaux wrote:
Obviously, the build part has to be well-tuned for the machinery to work,
but there is a lot of value here.
Ah yes, setuptools does have this. But this is specific to setuptools,
bare distutils does
On Thu, Jul 31, 2008 at 10:34:04AM -0400, Kevin Jacobs [EMAIL PROTECTED]
wrote:
The morale of this discussion, for me, is that just because _you_ don't
care about a particular aspect or feature, doesn't mean that others don't
or shouldn't. Your workarounds may not be viable for me
Andrew Dalke wrote:
If I had my way, remove things like (in numpy/__init__.py)
import linalg
import fft
import random
import ctypeslib
import ma
as a side benefit, this might help folks using py2exe, py2app and
friends -- as it stands all those sub-modules need
Christopher Barker wrote:
On my OS-X box (10.4.11, python2.5, numpy '1.1.1rc2'), it takes about 7
seconds to import numpy!
Hot or cold ? If hot, there is something horribly wrong with your setup.
On my macbook, it takes ~ 180 ms to to python -c import numpy, and ~
100 ms on linux (same
Stéfan van der Walt wrote:
No one is *waiting* for NumPy to start.
I am, and probably 10 times, a day, yes.
And it's a major issue for CGI, though maybe no one's using that anymore
anyway.
Just by answering this
e-mail I could have (and maybe should have) started NumPy three
hundred and
David Cournapeau wrote:
Christopher Barker wrote:
On my OS-X box (10.4.11, python2.5, numpy '1.1.1rc2'), it takes about 7
seconds to import numpy!
Hot or cold ? If hot, there is something horribly wrong with your setup.
hot -- it takes about 10 cold.
I've been wondering about that.
time
On Thu, 31 Jul 2008 10:12:22 -0700
Christopher Barker [EMAIL PROTECTED] wrote:
David Cournapeau wrote:
Christopher Barker wrote:
On my OS-X box (10.4.11, python2.5, numpy '1.1.1rc2'),
it takes about 7
seconds to import numpy!
Hot or cold ? If hot, there is something horribly wrong
with
On Thu, Jul 31, 2008 at 1:12 PM, Christopher Barker
[EMAIL PROTECTED]wrote:
David Cournapeau wrote:
Christopher Barker wrote:
On my OS-X box (10.4.11, python2.5, numpy '1.1.1rc2'), it takes about 7
seconds to import numpy!
Hot or cold ? If hot, there is something horribly wrong with
hot -- it takes about 10 cold.
I've been wondering about that.
time python -c import numpy
real0m8.383s
user0m0.320s
sys 0m7.805s
and similar results if run multiple times in a row.
What does python -c import sys; print sys.path say ?
Any idea what could be wrong? I have
On Thu, Jul 31, 2008 at 10:12:22AM -0700, Christopher Barker wrote:
I've been wondering about that.
time python -c import numpy
real0m8.383s
user0m0.320s
sys 0m7.805s
I don't know what is wrong, but this is plain wrong, unless you are on a
distant file system, or something
On Thu, Jul 31, 2008 at 07:46:20AM -0500, Nathan Bell wrote:
On Thu, Jul 31, 2008 at 7:31 AM, Hanni Ali [EMAIL PROTECTED] wrote:
I would just to highlight an alternate use of numpy to interactive use. We
have a cluster of machines which process tasks on an individual basis where
a master
On Thu, Jul 31, 2008 at 05:43, Andrew Dalke [EMAIL PROTECTED] wrote:
On Jul 31, 2008, at 12:03 PM, Robert Kern wrote:
But you still can't remove them since they are being used inside
numerictypes. That's why I labeled them internal utility functions
instead of leaving them with minimal
On Fri, Aug 1, 2008 at 5:02 AM, Robert Kern [EMAIL PROTECTED] wrote:
5) Proxy objects ... I would really like to avoid proxy objects. They
have caused fragility in the past.
One recurrent problem around import times optimization is that it is
some work to improve it, but it takes one line to
On Jul 4, 2008, at 2:22 PM, Andrew Dalke wrote:
[josiah:numpy/build/lib.macosx-10.3-fat-2.5] dalke% time python -c
'pass'
0.015u 0.042s 0:00.06 83.3% 0+0k 0+0io 0pf+0w
[josiah:numpy/build/lib.macosx-10.3-fat-2.5] dalke% time python -c
'import numpy'
0.084u 0.231s 0:00.33 93.9%
2008/7/30 Andrew Dalke [EMAIL PROTECTED]:
Based on those results I've been digging into the code trying to
figure out why numpy imports so many files, and at the same time I've
been trying to guess at the use case Robert Kern regards as typical
when he wrote:
Your use case isn't so
On Jul 30, 2008, at 10:59 PM, Stéfan van der Walt wrote:
I.e. most people don't start up NumPy all the time -- they import
NumPy, and then do some calculations, which typically take longer than
the import time.
Is that interactively, or is that through programs?
For a benefit of 0.03s, I
On Jul 30, 2008, at 10:51 PM, Alan McIntyre wrote:
I suppose it's necessary for providing the test() and bench()
functions in subpackages, but I that isn't a good reason to impose
upon all users the time required to set up numpy.testing.
I just posted this in my reply to Stéfan, but I'll say
On Thu, 2008-07-31 at 02:07 +0200, Andrew Dalke wrote:
On Jul 30, 2008, at 10:59 PM, Stéfan van der Walt wrote:
I.e. most people don't start up NumPy all the time -- they import
NumPy, and then do some calculations, which typically take longer than
the import time.
Is that interactively,
On Wed, Jul 30, 2008 at 8:19 PM, Andrew Dalke [EMAIL PROTECTED] wrote:
numpy defines
numpy.test
numpy.bench
and
numpy.testing.test
The two 'test's use the same implementation. This is a likely
unneeded duplication and one should be removed. The choice depends on
if people think the
On Jul 31, 2008, at 4:21 AM, Alan McIntyre wrote:
They actually do two different things; numpy.test() runs test for all
of numpy, and numpy.testing.test() runs tests for numpy.testing only.
There are similar functions in numpy.lib, numpy.core, etc.
Really? This is the code from
On Jul 3, 2008, at 9:06 AM, Robert Kern wrote:
Can you try the SVN trunk?
Sure. Though did you know it's not easy to find how to get numpy
from SVN? I had to go to the second page of Google, which linked to
someone's talk.
I expected to find a link to it at http://numpy.scipy.org/ .
Just
On Mon, Jun 30, 2008 at 18:32, Andrew Dalke [EMAIL PROTECTED] wrote:
Why does numpy/__init__.py need to import all of these other modules
and submodules? Any chance of cutting down on the number, in order
to improve startup costs?
Can you try the SVN trunk? In another thread (it must be numpy
Would it not be possible to import just the necessary module of numpy to
meet the necessary functionality of your application.
i.e.
import numpy.core
or whatever you're using
you could even do:
import numpy.core as numpy
I think, to simplify your code, I'm no expert though.
Hanni
2008/7/1
Hi,
IIRC, il you do import numpy.core as numpy, it starts by importing
numpy, so it will be even slower.
Matthieu
2008/7/1 Hanni Ali [EMAIL PROTECTED]:
Would it not be possible to import just the necessary module of numpy to
meet the necessary functionality of your application.
i.e.
You are correct, it appears to take slightly longer to import numpy.core and
longer again to import numpy.core as numpy
I should obviously check first in future.
Hanni
2008/7/1 Matthieu Brucher [EMAIL PROTECTED]:
Hi,
IIRC, il you do import numpy.core as numpy, it starts by importing
numpy,
2008/7/1 Hanni Ali [EMAIL PROTECTED]:
Would it not be possible to import just the necessary module of
numpy to
meet the necessary functionality of your application.
Matthieu Brucher responded:
IIRC, il you do import numpy.core as numpy, it starts by importing
numpy, so it will be even
(Trying again now that I'm subscribed. BTW, there's no link to the
subscription page from numpy.scipy.org .)
The initial 'import numpy' loads a huge number of modules, even when
I don't need them.
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)]
On Mon, Jun 30, 2008 at 18:32, Andrew Dalke [EMAIL PROTECTED] wrote:
(Trying again now that I'm subscribed. BTW, there's no link to the
subscription page from numpy.scipy.org .)
The initial 'import numpy' loads a huge number of modules, even when
I don't need them.
Python 2.5 (r25:51918,
On Jul 1, 2008, at 2:22 AM, Robert Kern wrote:
Your use case isn't so typical and so suffers on the import time
end of the
balance.
I'm working on my presentation for EuroSciPy. Isn't so typical
seems to be a good summary of my first slide. :)
Any chance of cutting down on the number,
55 matches
Mail list logo
