Re: [Numpy-discussion] Is numpy.test() supposed to be multithreaded?
On Mi, 2016-06-29 at 02:03 -0700, Nathaniel Smith wrote: > As a general rule I wouldn't worry too much about test speed. Speed > is > extremely dependent on exact workloads. And this is doubly so for > test > suites -- production workloads tend to do a small number of normal > things over and over, while a good test suite never does the same > thing twice and spends most of its time exercising weird edge > conditions. So unless your actual workload is running the numpy test > suite :-), it's probably not worth trying to track down. > Agreed, the test suit, and likely also the few tests which might take most time in the end, could be arbitrarily weird and skewed. I could for example imagine IO speed being a big factor. Also depending on system configuration (or numpy version) a different number of tests may be run sometimes. What might make somewhat more sense would be to compare some of the benchmarks `python runtests.py --bench` if you have airspeed velocity installed. While not extensive, a lot of those things at least do test more typical use cases. Though in any case I think the user should probably just test some other thing. - Sebastian > And yeah, numpy does not in general do automatic multithreading -- > the > only automatic multithreading you should see is when using linear > algebra functions (matrix multiply, eigenvalue calculations, etc.) > that dispatch to the BLAS. > > -n > > On Wed, Jun 29, 2016 at 12:07 AM, Ralf Gommers m> wrote: > > > > > > > > On Wed, Jun 29, 2016 at 3:27 AM, Chris Barker - NOAA Federal > > wrote: > > > > > > > > > > > > > > > > > > > > > Now the user is writing back to say, "my test code is fast now, > > > > but > > > > numpy.test() is still about three times slower than > > > server we > > > > don't manage>". When I watch htop as numpy.test() executes, > > > > sure enough, > > > > it's using one core > > > > > > > > > > > * if numpy.test() is supposed to be using multiple cores, why > > > > isn't it, > > > > when we've established with other test code that it's now using > > > > multiple > > > > cores? > > > > > > Some numpy.linalg functions (like np.dot) will be using multiple > > > cores, > > > but np.linalg.test() takes only ~1% of the time of the full test > > > suite. > > > Everything else will be running single core. So your observations > > > are not > > > surprising. > > > > > > > > > Though why it would run slower on one box than another comparable > > > box is a > > > mystery... > > > > Maybe just hardware config? I see a similar difference between how > > long the > > test suite runs on TravisCI vs my linux desktop (the latter is > > slower, > > surprisingly). > > > > Ralf > > > > > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@scipy.org > > https://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > signature.asc Description: This is a digitally signed message part ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is numpy.test() supposed to be multithreaded?
As a general rule I wouldn't worry too much about test speed. Speed is extremely dependent on exact workloads. And this is doubly so for test suites -- production workloads tend to do a small number of normal things over and over, while a good test suite never does the same thing twice and spends most of its time exercising weird edge conditions. So unless your actual workload is running the numpy test suite :-), it's probably not worth trying to track down. And yeah, numpy does not in general do automatic multithreading -- the only automatic multithreading you should see is when using linear algebra functions (matrix multiply, eigenvalue calculations, etc.) that dispatch to the BLAS. -n On Wed, Jun 29, 2016 at 12:07 AM, Ralf Gommers wrote: > > > On Wed, Jun 29, 2016 at 3:27 AM, Chris Barker - NOAA Federal > wrote: >> >> >>> >>> Now the user is writing back to say, "my test code is fast now, but >>> numpy.test() is still about three times slower than >> don't manage>". When I watch htop as numpy.test() executes, sure enough, >>> it's using one core >> >> >>> * if numpy.test() is supposed to be using multiple cores, why isn't it, >>> when we've established with other test code that it's now using multiple >>> cores? >> >> >> Some numpy.linalg functions (like np.dot) will be using multiple cores, >> but np.linalg.test() takes only ~1% of the time of the full test suite. >> Everything else will be running single core. So your observations are not >> surprising. >> >> >> Though why it would run slower on one box than another comparable box is a >> mystery... > > > Maybe just hardware config? I see a similar difference between how long the > test suite runs on TravisCI vs my linux desktop (the latter is slower, > surprisingly). > > Ralf > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > -- Nathaniel J. Smith -- https://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is numpy.test() supposed to be multithreaded?
On Wed, Jun 29, 2016 at 3:27 AM, Chris Barker - NOAA Federal < chris.bar...@noaa.gov> wrote: > > >> Now the user is writing back to say, "my test code is fast now, but >> numpy.test() is still about three times slower than > don't manage>". When I watch htop as numpy.test() executes, sure enough, >> it's using one core >> > > * if numpy.test() is supposed to be using multiple cores, why isn't it, >> when we've established with other test code that it's now using multiple >> cores? >> > > Some numpy.linalg functions (like np.dot) will be using multiple cores, > but np.linalg.test() takes only ~1% of the time of the full test suite. > Everything else will be running single core. So your observations are not > surprising. > > > Though why it would run slower on one box than another comparable box is a > mystery... > Maybe just hardware config? I see a similar difference between how long the test suite runs on TravisCI vs my linux desktop (the latter is slower, surprisingly). Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is numpy.test() supposed to be multithreaded?
> Now the user is writing back to say, "my test code is fast now, but > numpy.test() is still about three times slower than don't manage>". When I watch htop as numpy.test() executes, sure enough, > it's using one core > * if numpy.test() is supposed to be using multiple cores, why isn't it, > when we've established with other test code that it's now using multiple > cores? > Some numpy.linalg functions (like np.dot) will be using multiple cores, but np.linalg.test() takes only ~1% of the time of the full test suite. Everything else will be running single core. So your observations are not surprising. Though why it would run slower on one box than another comparable box is a mystery... -CHB ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is numpy.test() supposed to be multithreaded?
On Tue, Jun 28, 2016 at 10:36 PM, Michael Ward wrote: > Heya, I'm not a numbers guy, but I maintain servers for scientists and > researchers who are. Someone pointed out that our numpy installation on a > particular server was only using one core. I'm unaware of the who/how the > previous version of numpy/OpenBLAS were installed, so I installed them from > scratch, and confirmed that the users test code now runs on multiple cores > as expected, drastically increasing performance time. > > Now the user is writing back to say, "my test code is fast now, but > numpy.test() is still about three times slower than don't manage>". When I watch htop as numpy.test() executes, sure enough, > it's using one core. Now I'm not sure if that's the expected behavior or > not. Questions: > > * if numpy.test() is supposed to be using multiple cores, why isn't it, > when we've established with other test code that it's now using multiple > cores? > Some numpy.linalg functions (like np.dot) will be using multiple cores, but np.linalg.test() takes only ~1% of the time of the full test suite. Everything else will be running single core. So your observations are not surprising. Cheers, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Is numpy.test() supposed to be multithreaded?
Heya, I'm not a numbers guy, but I maintain servers for scientists and researchers who are. Someone pointed out that our numpy installation on a particular server was only using one core. I'm unaware of the who/how the previous version of numpy/OpenBLAS were installed, so I installed them from scratch, and confirmed that the users test code now runs on multiple cores as expected, drastically increasing performance time. Now the user is writing back to say, "my test code is fast now, but numpy.test() is still about three times slower than we don't manage>". When I watch htop as numpy.test() executes, sure enough, it's using one core. Now I'm not sure if that's the expected behavior or not. Questions: * if numpy.test() is supposed to be using multiple cores, why isn't it, when we've established with other test code that it's now using multiple cores? * if numpy.test() is not supposed to be using multiple cores, what could be the reason that the performance is drastically slower than another server with a comparable CPU, when the user's test code performs comparably? For what it's worth, the users "test" code which does run on multiple cores is as simple as: size=4000 a = np.random.random_sample((size,size)) b = np.random.random_sample((size,size)) x = np.dot(a,b) Whereas this uses only one core: numpy.test() --- OpenBLAS 0.2.18 was basically just compiled with "make", nothing special to it. Numpy 1.11.0 was installed from source (python setup.py install), using a site.cfg file to point numpy to the new OpenBLAS. Thanks, Mike ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
