On Tue, Oct 6, 2015 at 1:14 AM, Daπid wrote:
> One idea: what about creating a "parallel numpy"? There are a few
> algorithms that can benefit from parallelisation. This library would mimic
> Numpy's signature, and the user would be responsible for choosing the
> single
On 30 September 2015 at 18:20, Nathaniel Smith wrote:
> - parallel code in general is not very composable. If someone is calling a
> numpy operation from one thread, great, transparently using multiple
> threads internally is a win. If they're exploiting some higher-level
>
to:numpy-discussion@scipy.org>>
Cc:
Date: Fri, 2 Oct 2015 13:31:47 +0300
Subject: Re: [Numpy-discussion] Cython-based
OpenMP-accelerated quartic polynomial solver
On 02.10.2015 13:07, Daπid wrote:
On 2 October 2015 at 11:58, Juha Jeronen <juha
On 02/10/15 13:05, Daπid wrote:
Have you tried asking Python-dev for help with this? Hopefully they
would have some weight there.
It seems both GCC dev and Apple (for GCD and Accelerate) has taken a
similar stance on this. There is tiny set of functions the POSIX
standard demands should
On Mon, Oct 5, 2015 at 2:52 PM, Sturla Molden wrote:
> On 02/10/15 13:05, Daπid wrote:
>
>> Have you tried asking Python-dev for help with this? Hopefully they
>> would have some weight there.
>
> It seems both GCC dev and Apple (for GCD and Accelerate) has taken a
On Mon, Oct 5, 2015 at 3:05 PM, Nathaniel Smith wrote:
> On Mon, Oct 5, 2015 at 2:52 PM, Sturla Molden wrote:
>> On 02/10/15 13:05, Daπid wrote:
>>
>>> Have you tried asking Python-dev for help with this? Hopefully they
>>> would have some weight there.
On 01.10.2015 03:32, Sturla Molden wrote:
On 01/10/15 02:20, Juha Jeronen wrote:
Then again, the matter is further complicated by considering codes that
run on a single machine, versus codes that run on a cluster.Threads
being local to each node in a cluster,
You can run MPI programs on a
On 2 October 2015 at 11:58, Juha Jeronen wrote:
>
>>
> First version done and uploaded:
>
>
> https://yousource.it.jyu.fi/jjrandom2/miniprojects/trees/master/misc/polysolve_for_numpy
>
Small comment: now you are checking if the input is a scalar or a ndarray,
but it should
wrote:
> From: Juha Jeronen <juha.jero...@jyu.fi>
> To: Discussion of Numerical Python <numpy-discussion@scipy.org>
> Cc:
> Date: Fri, 2 Oct 2015 13:31:47 +0300
> Subject: Re: [Numpy-discussion] Cython-based OpenMP-accelerated quartic
> polynomial solver
> On 02.10.2015
ipy.org>
> wrote:
>
>> From: Juha Jeronen <juha.jero...@jyu.fi>
>> To: Discussion of Numerical Python <numpy-discussion@scipy.org>
>> Cc:
>> Date: Fri, 2 Oct 2015 13:31:47 +0300
>> Subject: Re: [Numpy-discussion] Cython-based OpenMP-accelerated quarti
On 02.10.2015 13:07, Daπid wrote:
On 2 October 2015 at 11:58, Juha Jeronen > wrote:
First version done and uploaded:
https://yousource.it.jyu.fi/jjrandom2/miniprojects/trees/master/misc/polysolve_for_numpy
Small comment: now you
On 01.10.2015 03:52, Sturla Molden wrote:
On 01/10/15 02:32, Juha Jeronen wrote:
Sounds good. Out of curiosity, are there any standard fork-safe
threadpools, or would this imply rolling our own?
You have to roll your own.
Basically use pthreads_atfork to register a callback that shuts down
On 1 October 2015 at 09:05, Nathaniel Smith wrote:
>
> >> - gcc + OpenMP on linux still breaks multiprocessing. There's a patch to
> >> fix this but they still haven't applied it; alternatively there's a
> >> workaround you can use in multiprocessing (not using fork mode), but
>
Juha Jeronen wrote:
> Mm. I've quite often run MPI locally (it's nice for multicore scientific
> computing on Python), but I had no idea that OpenMP had cluster
> implementations. Thanks for the tip.
Intel has been selling one, I think there are others too.
OpenMP has a
Sturla Molden wrote:
> OpenMP has a flush pragma for synchronizing shared variables. This means
> that OpenMP is not restricted to shared memory hardware. A "pragma omp
> flush" can just as well invoke some IPC mechanism, even network
> communication.
By the way, while
Sturla Molden wrote:
> Cython actually requires that there is a shared address space, and it
> invokes something that strictly speaking has undefined behavior under the
> OpenMP standard. So thus, a prange block in Cython is expected to work
> correctly on a laptop with
On Wed, Sep 30, 2015 at 11:54 PM, Daπid wrote:
>
>
> On 30 September 2015 at 18:20, Nathaniel Smith wrote:
>>
>> On Sep 30, 2015 2:28 AM, "Daπid" wrote:
>> [...]
>> > Is there a nice way to ship both versions? After all, most
>> >
On 30 September 2015 at 18:20, Nathaniel Smith wrote:
> On Sep 30, 2015 2:28 AM, "Daπid" wrote:
> [...]
> > Is there a nice way to ship both versions? After all, most
> implementations of BLAS and friends do spawn OpenMP threads, so I don't
> think it
After, I agree with you.
2015-09-30 18:14 GMT+01:00 Robert Kern :
> On Wed, Sep 30, 2015 at 10:35 AM, Matthieu Brucher
> wrote:
>>
>> Yes, obviously, the code has NR parts, so it can't be licensed as BSD
>> as it is...
>
> It's not obvious to
On Wed, Sep 30, 2015 at 10:35 AM, Matthieu Brucher <
matthieu.bruc...@gmail.com> wrote:
>
> Yes, obviously, the code has NR parts, so it can't be licensed as BSD
> as it is...
It's not obvious to me, especially after Juha's further clarifications.
--
Robert Kern
Hi,
On 30.09.2015 03:48, Chris Barker - NOAA Federal wrote:
This sounds pretty cool -- and I've had a use case. So it would be
nice to get into Numpy.
But: I doubt we'd want OpenMP dependence in Numpy itself.
It is indeed a good point not to add new dependencies for a small
feature such as
Hi,
On 30.09.2015 04:37, Charles R Harris wrote:
On Tue, Sep 29, 2015 at 6:48 PM, Chris Barker - NOAA Federal
> wrote:
This sounds pretty cool -- and I've had a use case. So it would be
nice to get into Numpy.
But: I doubt
Hi,
What qualifies as an NR part?
(See my previous message concerning the references; NR is not the only
source which has these algorithms. Again, the code itself is original to
this solver, I haven't even looked at the code provided with NR.)
-J
Juha Jeronen jyu.fi> writes:
> I recently developed a Cython-based, OpenMP-accelerated quartic (and
> cubic, quadratic) polynomial solver to address a personal research need
> for quickly solving a very large number of independent low-degree
> polynomial equations for both real and complex
On 30 September 2015 at 10:20, Juha Jeronen wrote:
>
> Are we putting Cuthon in Numpy now? I lost track.
>>
>
> I thought so? But then again, I'm just a user :)
As of this moment, there are three Cython modules in Numpy, so yes.
Releases ship the C generated modules, so it
Yes, obviously, the code has NR parts, so it can't be licensed as BSD
as it is...
Matthieu
2015-09-30 2:37 GMT+01:00 Charles R Harris :
>
>
> On Tue, Sep 29, 2015 at 6:48 PM, Chris Barker - NOAA Federal
> wrote:
>>
>> This sounds pretty cool --
On 30.09.2015 19:20, Chris Barker wrote:
On Wed, Sep 30, 2015 at 6:57 AM, Nathan Goldbaum
> wrote:
Note however that with the current version of the code, not
having OpenMP will severely limit the performance, especially
On 30.09.2015 19:20, Nathaniel Smith wrote:
The challenges to providing transparent multithreading in numpy
generally are:
- gcc + OpenMP on linux still breaks multiprocessing. There's a patch
to fix this but they still haven't applied it; alternatively there's a
workaround you can use in
On 30/09/15 18:20, Chris Barker wrote:
We'd need a run-time check.
We need to amend the compiler classes in numpy.distutils with OpenMP
relevant information (compiler flags and libraries).
The OpenMP support libraries must be statically linked.
Sturla
On 30/09/15 18:20, Chris Barker wrote:
python threads with nogil?
This is often the easiest, particularly if we construct a fork-safe
threadpool.
Sturla
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
On 01/10/15 02:20, Juha Jeronen wrote:
Then again, the matter is further complicated by considering codes that
run on a single machine, versus codes that run on a cluster.Threads
being local to each node in a cluster,
You can run MPI programs on a single machine and you get OpenMP
On 01.10.2015 01:04, Sturla Molden wrote:
There are two principal problems with using OpenMP in NumPy:
One is that the GNU OpenMP threadpool is not fork-safe, and can break
multiprocessing on some platforms (e.g. when using Python 2.7 on
Linux). Anything that uses GCD has this nasty effect on
On 30/09/15 11:27, Daπid wrote:
Is there a nice way to ship both versions? After all, most
implementations of BLAS and friends do spawn OpenMP threads, so I don't
think it would be outrageous to take advantage of it in more places;
Some do, others don't.
ACML uses OpenMP.
GotoBLAS uses
On 30.09.2015 13:21, Pauli Virtanen wrote:
Juha Jeronen jyu.fi> writes:
I recently developed a Cython-based, OpenMP-accelerated quartic (and
cubic, quadratic) polynomial solver to address a personal research need
for quickly solving a very large number of independent low-degree
polynomial
On 30/09/15 18:20, Nathaniel Smith wrote:
This is incorrect -- the only common implementation of BLAS that uses
*OpenMP* threads is OpenBLAS,
MKL and ACML also use OpenMP.
Sturla
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
On 30/09/15 15:57, Nathan Goldbaum wrote:
Basically, just try to compile a simple OpenMP test program in a
subprocess. If that succeeds, then great, we can add -fopenmp as a
compilation flag. If not, don't do that.
Unless you use GCC on Linux, it will be more complex than that. I.e. do
you
On 01/10/15 02:32, Juha Jeronen wrote:
Sounds good. Out of curiosity, are there any standard fork-safe
threadpools, or would this imply rolling our own?
You have to roll your own.
Basically use pthreads_atfork to register a callback that shuts down the
threadpool before a fork and another
On Wed, Sep 30, 2015 at 3:20 AM, Juha Jeronen wrote:
> Hi,
>
> On 30.09.2015 03:48, Chris Barker - NOAA Federal wrote:
>
>> This sounds pretty cool -- and I've had a use case. So it would be
>> nice to get into Numpy.
>>
>> But: I doubt we'd want OpenMP dependence in Numpy
On Sep 30, 2015 2:28 AM, "Daπid" wrote:
[...]
> Is there a nice way to ship both versions? After all, most
implementations of BLAS and friends do spawn OpenMP threads, so I don't
think it would be outrageous to take advantage of it in more places;
provided there is a nice
On Wed, Sep 30, 2015 at 6:57 AM, Nathan Goldbaum
wrote:
> Note however that with the current version of the code, not having OpenMP
>> will severely limit the performance, especially on quad-core machines,
>> since an important factor in the speed comes from the parallel
Hi all,
I recently developed a Cython-based, OpenMP-accelerated quartic (and
cubic, quadratic) polynomial solver to address a personal research need
for quickly solving a very large number of independent low-degree
polynomial equations for both real and complex coefficients.
For example, on
The algorithms for cubics and quadratics come from Numerical Recipes
(3rd ed.), and the quartic problem is internally reduced to a cubic and
two quadratics, using well-known standard tricks.Nice, wll documented code.
Just to be sure you are on safe ground, you took the *algorithms*
This sounds pretty cool -- and I've had a use case. So it would be
nice to get into Numpy.
But: I doubt we'd want OpenMP dependence in Numpy itself.
But maybe a pure Cython non-MP version?
Are we putting Cuthon in Numpy now? I lost track.
-CHB
Sent from my iPhone
> On Sep 29, 2015, at 7:35
On Tue, Sep 29, 2015 at 6:48 PM, Chris Barker - NOAA Federal <
chris.bar...@noaa.gov> wrote:
> This sounds pretty cool -- and I've had a use case. So it would be
> nice to get into Numpy.
>
> But: I doubt we'd want OpenMP dependence in Numpy itself.
>
> But maybe a pure Cython non-MP version?
>
>
Hi,
On 29.09.2015 19:16, Matti Picus wrote:
The algorithms for cubics and quadratics come from Numerical Recipes
(3rd ed.), and the quartic problem is internally reduced to a cubic and
two quadratics, using well-known standard tricks.
Nice, wll documented code.
Just to be sure you are on safe
45 matches
Mail list logo