Re: [Numpy-discussion] How to use user input as equation directly

2016-10-27 Thread Benjamin Root 
"only be used by engineers/scientists for research"

Famous last words. I know plenty of scientists who would love to "do
research" with an exposed eval(). Full disclosure, I personally added a
security hole into matplotlib thinking I covered all my bases in protecting
an eval() statement.

Ben Root

On Thu, Oct 27, 2016 at 4:21 PM, djxvillain  wrote:

> This will not be a public product and will only be used by other
> engineers/scientists for research.  I don't think security should be a huge
> issue, but I appreciate your input and concern for the quality of my code.
>
>
>
> --
> View this message in context: http://numpy-discussion.10968.
> n7.nabble.com/How-to-use-user-input-as-equation-directly-
> tp43665p43670.html
> Sent from the Numpy-discussion mailing list archive at Nabble.com.
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to use user input as equation directly

2016-10-27 Thread djxvillain 
This will not be a public product and will only be used by other
engineers/scientists for research.  I don't think security should be a huge
issue, but I appreciate your input and concern for the quality of my code.



--
View this message in context: 
http://numpy-discussion.10968.n7.nabble.com/How-to-use-user-input-as-equation-directly-tp43665p43670.html
Sent from the Numpy-discussion mailing list archive at Nabble.com.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to use user input as equation directly

2016-10-27 Thread Benjamin Root 
Perhaps the numexpr package might be safer? Not exactly meant for this
situation (meant for optimizations), but the evaluator is pretty darn safe.

Ben Root

On Thu, Oct 27, 2016 at 5:33 PM, John Ladasky  wrote:

> This isn't just a Numpy issue.  You are interested in Python's eval().
>
> Keep in mind that any programming language that blurs the line between
> code and data (many do not) has a potential security vulnerability.  What
> if your user doesn't type
>
> "x = 2*np.sin(2*np.pi*44100*t+np.pi/2)"
>
> but instead types this:
>
> "import os ; os.remove('/home')"
>
> I do NOT recommend that you eval() the second statement.
>
> You can try to write code which traps unwanted input before you eval()
> it.  It's apparently quite hard to stop everything bad from getting through.
>
>
> On Thu, Oct 27, 2016 at 12:58 PM, djxvillain  wrote:
>
>> Hello all,
>>
>> I am an electrical engineer and new to numpy.  I need the ability to take
>> in
>> user input, and use that input as a variable.  For example:
>>
>> t = input('enter t: ')
>> x = input('enter x: ')
>>
>> I need the user to be able to enter something like x =
>> 2*np.sin(2*np.pi*44100*t+np.pi/2) and it be the same as if they just
>> typed
>> it in the .py file.  There's no clean way to cast or evaluate it that I've
>> found.
>>
>> I could make a function to parse this string character by character, but I
>> figured this is probably a common problem and someone else has probably
>> figured it out and created an object for it.  I can't find a library that
>> does it though.
>>
>> If I can provide any more information please let me know.  Thank you in
>> advance for your help.
>>
>>
>>
>> --
>> View this message in context: http://numpy-discussion.10968.
>> n7.nabble.com/How-to-use-user-input-as-equation-directly-tp43665.html
>> Sent from the Numpy-discussion mailing list archive at Nabble.com.
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
>
> --
> *John J. Ladasky Jr., Ph.D.*
> *Research Scientist*
> *International Technological University*
> *2711 N. First St, San Jose, CA 95134 USA*
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to use user input as equation directly

2016-10-27 Thread John Ladasky 
This isn't just a Numpy issue.  You are interested in Python's eval().

Keep in mind that any programming language that blurs the line between code
and data (many do not) has a potential security vulnerability.  What if
your user doesn't type

"x = 2*np.sin(2*np.pi*44100*t+np.pi/2)"

but instead types this:

"import os ; os.remove('/home')"

I do NOT recommend that you eval() the second statement.

You can try to write code which traps unwanted input before you eval() it.
It's apparently quite hard to stop everything bad from getting through.


On Thu, Oct 27, 2016 at 12:58 PM, djxvillain  wrote:

> Hello all,
>
> I am an electrical engineer and new to numpy.  I need the ability to take
> in
> user input, and use that input as a variable.  For example:
>
> t = input('enter t: ')
> x = input('enter x: ')
>
> I need the user to be able to enter something like x =
> 2*np.sin(2*np.pi*44100*t+np.pi/2) and it be the same as if they just typed
> it in the .py file.  There's no clean way to cast or evaluate it that I've
> found.
>
> I could make a function to parse this string character by character, but I
> figured this is probably a common problem and someone else has probably
> figured it out and created an object for it.  I can't find a library that
> does it though.
>
> If I can provide any more information please let me know.  Thank you in
> advance for your help.
>
>
>
> --
> View this message in context: http://numpy-discussion.10968.
> n7.nabble.com/How-to-use-user-input-as-equation-directly-tp43665.html
> Sent from the Numpy-discussion mailing list archive at Nabble.com.
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
*John J. Ladasky Jr., Ph.D.*
*Research Scientist*
*International Technological University*
*2711 N. First St, San Jose, CA 95134 USA*
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to use user input as equation directly

2016-10-27 Thread djxvillain 
That worked perfectly.  I've been googling how to do this, I guess I didn't
phrase it correctly.  Thank you very much.  You just saved me a ton of time.



--
View this message in context: 
http://numpy-discussion.10968.n7.nabble.com/How-to-use-user-input-as-equation-directly-tp43665p43667.html
Sent from the Numpy-discussion mailing list archive at Nabble.com.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to use user input as equation directly

2016-10-27 Thread Ryan May 
On Thu, Oct 27, 2016 at 1:58 PM, djxvillain  wrote:

> Hello all,
>
> I am an electrical engineer and new to numpy.  I need the ability to take
> in
> user input, and use that input as a variable.  For example:
>
> t = input('enter t: ')
> x = input('enter x: ')
>
> I need the user to be able to enter something like x =
> 2*np.sin(2*np.pi*44100*t+np.pi/2) and it be the same as if they just typed
> it in the .py file.  There's no clean way to cast or evaluate it that I've
> found.
>

Are you aware of Python's eval function:
https://docs.python.org/3/library/functions.html#eval

?

Ryan

-- 
Ryan May
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] How to use user input as equation directly

2016-10-27 Thread djxvillain 
Hello all,

I am an electrical engineer and new to numpy.  I need the ability to take in
user input, and use that input as a variable.  For example:

t = input('enter t: ')
x = input('enter x: ')

I need the user to be able to enter something like x =
2*np.sin(2*np.pi*44100*t+np.pi/2) and it be the same as if they just typed
it in the .py file.  There's no clean way to cast or evaluate it that I've
found.

I could make a function to parse this string character by character, but I
figured this is probably a common problem and someone else has probably
figured it out and created an object for it.  I can't find a library that
does it though.

If I can provide any more information please let me know.  Thank you in
advance for your help.



--
View this message in context: 
http://numpy-discussion.10968.n7.nabble.com/How-to-use-user-input-as-equation-directly-tp43665.html
Sent from the Numpy-discussion mailing list archive at Nabble.com.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Added atleast_nd, request for clarification/cleanup of atleast_3d

2016-10-27 Thread Joseph Fox-Rabinovitz 
Hi,

I would like to revitalize the discussion on including PR#7804 (atleast_nd
function) at Stephan Hoyer's request. atleast_nd has come up as a
convenient workaround for #8206 (adding padding options to diff) to be able
to do broadcasting with the required dimensions reversed.

Regards,

-Joe



On Mon, Jul 11, 2016 at 10:41 AM, Joseph Fox-Rabinovitz <
jfoxrabinov...@gmail.com> wrote:

> I would like to follow up on my original PR (7804). While there
> appears to be some debate as to whether the PR is numpy material to
> begin with, there do not appear to be any technical issues with it. To
> make the decision more straightforward, I factored out the
> non-controversial bug fixes to masked arrays into PR #7823, along with
> their regression tests. This way, the original enhancement can be
> closed or left hanging indefinitely, (even though I hope neither
> happens). PR 7804 still has the bug fixes duplicated in it.
>
> Regards,
>
> -Joe
>
>
> On Thu, Jul 7, 2016 at 9:11 AM, Joseph Fox-Rabinovitz
>  wrote:
> > On Thu, Jul 7, 2016 at 4:34 AM, Sebastian Berg
> >  wrote:
> >> On Mi, 2016-07-06 at 15:30 -0400, Benjamin Root wrote:
> >>> I don't see how one could define a spec that would take an arbitrary
> >>> array of indices at which to place new dimensions. By definition, you
> >>>
> >>
> >> You just give a reordered range, so that (1, 0, 2) would be the current
> >> 3D version. If 1D, fill in `1` and `2`, if 2D, fill in only `2` (0D,
> >> add everything of course).
> >
> > I was originally thinking (-1, 0) for the 2D case. Just go along the
> > list and fill as many dims as necessary. Your way is much better since
> > it does not require a different operation for positive and negative
> > indices.
> >
> >> However, I have my doubts that it is actually easier to understand then
> >> to write yourself ;).
> >
> > A dictionary or ragged list would be better for that: either {1: (1,
> > 0), 2: (2,)} or [(1, 0), (2,)]. The first is more clear since the
> > index in the list is the starting ndim - 1.
> >
> >>
> >> - Sebastian
> >>
> >>
> >>> don't know how many dimensions are going to be added. If you knew,
> >>> then you wouldn't be calling this function. I can only imagine simple
> >>> rules such as 'left' or 'right' or maybe something akin to what
> >>> at_least3d() implements.
> >>>
> >>> On Wed, Jul 6, 2016 at 3:20 PM, Joseph Fox-Rabinovitz  >>> @gmail.com> wrote:
> >>> > On Wed, Jul 6, 2016 at 2:57 PM, Eric Firing 
> >>> > wrote:
> >>> > > On 2016/07/06 8:25 AM, Benjamin Root wrote:
> >>> > >>
> >>> > >> I wouldn't have the keyword be "where", as that collides with
> >>> > the notion
> >>> > >> of "where" elsewhere in numpy.
> >>> > >
> >>> > >
> >>> > > Agreed.  Maybe "side"?
> >>> >
> >>> > I have tentatively changed it to "pos". The reason that I don't
> >>> > like
> >>> > "side" is that it implies only a subset of the possible ways that
> >>> > that
> >>> > the position of the new dimensions can be specified. The current
> >>> > implementation only puts things on one side or the other, but I
> >>> > have
> >>> > considered also allowing an array of indices at which to place new
> >>> > dimensions, and/or a dictionary keyed by the starting ndims. I do
> >>> > not
> >>> > think "side" would be appropriate for these extended cases, even if
> >>> > they are very unlikely to ever materialize.
> >>> >
> >>> > -Joe
> >>> >
> >>> > > (I find atleast_1d and atleast_2d to be very helpful for handling
> >>> > inputs, as
> >>> > > Ben noted; I'm skeptical as to the value of atleast_3d and
> >>> > atleast_nd.)
> >>> > >
> >>> > > Eric
> >>> > >
> >>> > > ___
> >>> > > NumPy-Discussion mailing list
> >>> > > NumPy-Discussion@scipy.org
> >>> > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> >>> > ___
> >>> > NumPy-Discussion mailing list
> >>> > NumPy-Discussion@scipy.org
> >>> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> >>> >
> >>> ___
> >>> NumPy-Discussion mailing list
> >>> NumPy-Discussion@scipy.org
> >>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
> >>
> >> ___
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion@scipy.org
> >> https://mail.scipy.org/mailman/listinfo/numpy-discussion
> >>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Robert Kern 
On Thu, Oct 27, 2016 at 10:45 AM, Todd  wrote:
>
> On Thu, Oct 27, 2016 at 12:12 PM, Nathaniel Smith  wrote:
>>
>> Ever notice how Anaconda doesn't provide pyfftw? They can't legally ship
both MKL and pyfftw, and they picked MKL.
>
> Anaconda does ship GPL code [1].  They even ship GPL code that depends on
numpy, such as cvxcanon and pystan, and there doesn't seem to be anything
that prevents me from installing them alongside the MKL version of numpy.
So I don't see how it would be any different for pyfftw.

I think we've exhausted the relevance of this tangent to Oleksander's
contributions.

--
Robert Kern
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Todd 
On Thu, Oct 27, 2016 at 12:12 PM, Nathaniel Smith  wrote:

> Ever notice how Anaconda doesn't provide pyfftw? They can't legally ship
> both MKL and pyfftw, and they picked MKL.


Anaconda does ship GPL code [1].  They even ship GPL code that depends on
numpy, such as cvxcanon and pystan, and there doesn't seem to be anything
that prevents me from installing them alongside the MKL version of numpy.
So I don't see how it would be any different for pyfftw.

[1] https://docs.continuum.io/anaconda/pkg-docs
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Nathaniel Smith 
On Oct 27, 2016 8:42 AM, "Robert McLeod"  wrote:
>
> Releasing NumPy under GPL would make it incompatible with SciPy, which
may be _slightly_ inconvenient to the scientific Python community:
>
> https://scipy.github.io/old-wiki/pages/License_Compatibility.html
>
> https://mail.scipy.org/pipermail/scipy-dev/2013-August/019149.html

There's 0 chance that numpy is going to switch to the GPL in general, so
please don't panic. Also, you're misunderstanding license compatibility, so
let's back up a step :-).

The discussion was about whether numpy might potentially, at some
unspecified future date, be available with *optional* GPL code. A numpy
build with optional GPL bits included would be similar to how the numpy
builds that many people use which that are linked to MKL, and thus subject
to MKL's license terms. In both cases the license is no longer numpy's
regular bsd, but has these extra bits added. Neither changes the
availability of bsd-licensed numpy; they just give another option.

And, both numpy+GPL-bits and numpy+MKL-bits are/would be license
*compatible* with scipy in the sense that matters to end users: you can
absolutely use and distribute numpy+(pick one of the above)+scipy together,
and the licenses are happy to allow that.

The sense in which they're both *in*compatible with scipy is just that if
you want to *add code to scipy itself*, then that code can't be GPL like
pyfftw, or proprietary like MKL, because the scipy devs have decided that
they don't want to allow that. That's a decision they've made for good
reasons, but it isn't a legal inevitability, and it doesn't stop *you* from
using and distributing scipy and GPL code together, or scipy and
proprietary code together.

(The real license incompatibility is between GPL and proprietary. Either
one can be mixed with BSD, but they can't be mixed with each other and then
distributed. Ever notice how Anaconda doesn't provide pyfftw? They can't
legally ship both MKL and pyfftw, and they picked MKL. Even then, though,
this license restriction only applies to software distributors: if you as
an end user go and install MKL and pyfftw together in the privacy of your
own cluster, then that's also totally legal.)

-n
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Julian Taylor 
As I understand it the wiki is talking about including code in 
numpy/scipy itself, all code in numpy and scipy must be permissively 
licensed so it is easy to reason about when building your binaries.


The license of the binaries produced from the code is a different 
matter, which at that time didn't really exist as we didn't distribute 
binaries at all (except for windows).


A GPL licensed binary containing numpy is perfectly compatible with 
SciPy. It may not be compatible with some other component which has an 
actually incompatible license (e.g. anything you cannot distribute the 
source of as required by the GPL).
I it is not numpy that is GPL licensed it is the restriction of another 
component in the binary distribution that makes the full product adhere 
to the most restrictive license
But numpy itself is always permissive, the distributor can always build 
a permissive numpy binary without the viral component in it.



On 10/27/2016 05:42 PM, Robert McLeod wrote:

Releasing NumPy under GPL would make it incompatible with SciPy, which
may be _slightly_ inconvenient to the scientific Python community:

https://scipy.github.io/old-wiki/pages/License_Compatibility.html

https://mail.scipy.org/pipermail/scipy-dev/2013-August/019149.html

Robert

On Thu, Oct 27, 2016 at 5:14 PM, Julian Taylor
>
wrote:

On 10/27/2016 04:52 PM, Todd wrote:

On Thu, Oct 27, 2016 at 10:43 AM, Julian Taylor

>>
wrote:

On 10/27/2016 04:30 PM, Todd wrote:

On Thu, Oct 27, 2016 at 4:25 AM, Ralf Gommers

>
 
>


>

>>] *On Behalf Of *Todd
*Sent:* Wednesday, October 26, 2016 4:04 PM
*To:* Discussion of Numerical Python

>


>

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Todd 
It would still be compatible with SciPy, it would "just" mean that SciPy
(and anything else that uses numpy) would be effectively GPL.

On Thu, Oct 27, 2016 at 11:42 AM, Robert McLeod 
wrote:

> Releasing NumPy under GPL would make it incompatible with SciPy, which may
> be _slightly_ inconvenient to the scientific Python community:
>
> https://scipy.github.io/old-wiki/pages/License_Compatibility.html
>
> https://mail.scipy.org/pipermail/scipy-dev/2013-August/019149.html
>
> Robert
>
> On Thu, Oct 27, 2016 at 5:14 PM, Julian Taylor <
> jtaylor.deb...@googlemail.com> wrote:
>
>> On 10/27/2016 04:52 PM, Todd wrote:
>>
>>> On Thu, Oct 27, 2016 at 10:43 AM, Julian Taylor
>>> >
>>> wrote:
>>>
>>> On 10/27/2016 04:30 PM, Todd wrote:
>>>
>>> On Thu, Oct 27, 2016 at 4:25 AM, Ralf Gommers
>>> 
>>> >>
>>> wrote:
>>>
>>>
>>> On Thu, Oct 27, 2016 at 10:25 AM, Pavlyk, Oleksandr
>>> >> 
>>> >> >> wrote:
>>>
>>> Please see responses inline.
>>>
>>>
>>>
>>> *From:*NumPy-Discussion
>>> [mailto:numpy-discussion-boun...@scipy.org
>>> 
>>> >> >] *On Behalf Of
>>> *Todd
>>> *Sent:* Wednesday, October 26, 2016 4:04 PM
>>> *To:* Discussion of Numerical Python
>>> 
>>> >> >>
>>> *Subject:* Re: [Numpy-discussion] Intel random number
>>> package
>>>
>>>
>>>
>>>
>>> On Wed, Oct 26, 2016 at 4:30 PM, Pavlyk, Oleksandr
>>> >> 
>>> >>
>>> >>
>>> wrote:
>>>
>>> Another point already raised by Nathaniel is that for
>>> numpy's randomness ideally should provide a way to
>>> override
>>> default algorithm for sampling from a particular
>>> distribution.  For example RandomState object that
>>> implements PCG may rely on default
>>> acceptance-rejection
>>> algorithm for sampling from Gamma, while the
>>> RandomState
>>> object that provides interface to MKL might want to
>>> call
>>> into MKL directly.
>>>
>>>
>>>
>>> The approach that pyfftw uses at least for scipy, which
>>> may also
>>> work here, is that you can monkey-patch the
>>> scipy.fftpack module
>>> at runtime, replacing it with pyfftw's drop-in
>>> replacement.
>>> scipy then proceeds to use pyfftw instead of its built-in
>>> fftpack implementation.  Might such an approach work
>>> here?
>>> Users can either use this alternative randomstate
>>> replacement
>>> directly, or they can replace numpy's with it at runtime
>>> and
>>> numpy will then proceed to use the alternative.
>>>
>>>
>>> The only reason that pyfftw uses monkeypatching is that the
>>> better
>>> approach is not possible due to license constraints with
>>> FFTW (it's
>>> GPL).
>>>
>>>
>>> Yes, that is exactly why I brought it up.  Better approaches are
>>> also
>>> not possible with MKL due to license constraints.  It is a very
>>> similar
>>> situation overall.
>>>
>>>
>>> Its not that similar, the better approach is certainly possible with
>>> FFTW, the GPL is compatible with numpys license. It is only a
>>> concern users of binary distributions. Nobody provided the code to
>>> use fftw yet, but it would certainly be accepted.
>>>
>>>
>>> Although it is technically compatible, it would make numpy effectively
>>> GPL.  Suggestions for this have been explicitly rejected on these
>>> grounds [1]
>>>
>>> [1] https://github.com/numpy/numpy/issues/3485
>>>
>>>
>> Yes it would make numpy GPL, but that is not a concern for a lot of
>> users. Users for who it is a problem can still use the non-GPL version.
>> A more interesting debate is whether our binary wheels should then be GPL
>> wheels by default or not. Probably not, but that is something that

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Robert McLeod 
Releasing NumPy under GPL would make it incompatible with SciPy, which may
be _slightly_ inconvenient to the scientific Python community:

https://scipy.github.io/old-wiki/pages/License_Compatibility.html

https://mail.scipy.org/pipermail/scipy-dev/2013-August/019149.html

Robert

On Thu, Oct 27, 2016 at 5:14 PM, Julian Taylor <
jtaylor.deb...@googlemail.com> wrote:

> On 10/27/2016 04:52 PM, Todd wrote:
>
>> On Thu, Oct 27, 2016 at 10:43 AM, Julian Taylor
>> >
>> wrote:
>>
>> On 10/27/2016 04:30 PM, Todd wrote:
>>
>> On Thu, Oct 27, 2016 at 4:25 AM, Ralf Gommers
>> 
>> >>
>> wrote:
>>
>>
>> On Thu, Oct 27, 2016 at 10:25 AM, Pavlyk, Oleksandr
>> > 
>> > >> wrote:
>>
>> Please see responses inline.
>>
>>
>>
>> *From:*NumPy-Discussion
>> [mailto:numpy-discussion-boun...@scipy.org
>> 
>> > >] *On Behalf Of *Todd
>> *Sent:* Wednesday, October 26, 2016 4:04 PM
>> *To:* Discussion of Numerical Python
>> 
>> > >>
>> *Subject:* Re: [Numpy-discussion] Intel random number
>> package
>>
>>
>>
>>
>> On Wed, Oct 26, 2016 at 4:30 PM, Pavlyk, Oleksandr
>> > 
>> >
>> >>
>> wrote:
>>
>> Another point already raised by Nathaniel is that for
>> numpy's randomness ideally should provide a way to
>> override
>> default algorithm for sampling from a particular
>> distribution.  For example RandomState object that
>> implements PCG may rely on default
>> acceptance-rejection
>> algorithm for sampling from Gamma, while the
>> RandomState
>> object that provides interface to MKL might want to
>> call
>> into MKL directly.
>>
>>
>>
>> The approach that pyfftw uses at least for scipy, which
>> may also
>> work here, is that you can monkey-patch the
>> scipy.fftpack module
>> at runtime, replacing it with pyfftw's drop-in
>> replacement.
>> scipy then proceeds to use pyfftw instead of its built-in
>> fftpack implementation.  Might such an approach work here?
>> Users can either use this alternative randomstate
>> replacement
>> directly, or they can replace numpy's with it at runtime
>> and
>> numpy will then proceed to use the alternative.
>>
>>
>> The only reason that pyfftw uses monkeypatching is that the
>> better
>> approach is not possible due to license constraints with
>> FFTW (it's
>> GPL).
>>
>>
>> Yes, that is exactly why I brought it up.  Better approaches are
>> also
>> not possible with MKL due to license constraints.  It is a very
>> similar
>> situation overall.
>>
>>
>> Its not that similar, the better approach is certainly possible with
>> FFTW, the GPL is compatible with numpys license. It is only a
>> concern users of binary distributions. Nobody provided the code to
>> use fftw yet, but it would certainly be accepted.
>>
>>
>> Although it is technically compatible, it would make numpy effectively
>> GPL.  Suggestions for this have been explicitly rejected on these
>> grounds [1]
>>
>> [1] https://github.com/numpy/numpy/issues/3485
>>
>>
> Yes it would make numpy GPL, but that is not a concern for a lot of users.
> Users for who it is a problem can still use the non-GPL version.
> A more interesting debate is whether our binary wheels should then be GPL
> wheels by default or not. Probably not, but that is something that should
> be discussed when its an actual issue.
>
> But to clarify what I said, it would be accepted if the value it provides
> is sufficient compared to the code maintenance it adds. Given that pyfftw
> already exists the value is probably relatively small, but personally I'd
> still be interested in code that allows switching the fft backend as that
>

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Julian Taylor 

On 10/27/2016 04:52 PM, Todd wrote:

On Thu, Oct 27, 2016 at 10:43 AM, Julian Taylor
>
wrote:

On 10/27/2016 04:30 PM, Todd wrote:

On Thu, Oct 27, 2016 at 4:25 AM, Ralf Gommers

>>
wrote:


On Thu, Oct 27, 2016 at 10:25 AM, Pavlyk, Oleksandr

>> wrote:

Please see responses inline.



*From:*NumPy-Discussion
[mailto:numpy-discussion-boun...@scipy.org

>] *On Behalf Of *Todd
*Sent:* Wednesday, October 26, 2016 4:04 PM
*To:* Discussion of Numerical Python

>>
*Subject:* Re: [Numpy-discussion] Intel random number
package




On Wed, Oct 26, 2016 at 4:30 PM, Pavlyk, Oleksandr

>>
wrote:

Another point already raised by Nathaniel is that for
numpy's randomness ideally should provide a way to
override
default algorithm for sampling from a particular
distribution.  For example RandomState object that
implements PCG may rely on default acceptance-rejection
algorithm for sampling from Gamma, while the RandomState
object that provides interface to MKL might want to call
into MKL directly.



The approach that pyfftw uses at least for scipy, which
may also
work here, is that you can monkey-patch the
scipy.fftpack module
at runtime, replacing it with pyfftw's drop-in replacement.
scipy then proceeds to use pyfftw instead of its built-in
fftpack implementation.  Might such an approach work here?
Users can either use this alternative randomstate
replacement
directly, or they can replace numpy's with it at runtime and
numpy will then proceed to use the alternative.


The only reason that pyfftw uses monkeypatching is that the
better
approach is not possible due to license constraints with
FFTW (it's
GPL).


Yes, that is exactly why I brought it up.  Better approaches are
also
not possible with MKL due to license constraints.  It is a very
similar
situation overall.


Its not that similar, the better approach is certainly possible with
FFTW, the GPL is compatible with numpys license. It is only a
concern users of binary distributions. Nobody provided the code to
use fftw yet, but it would certainly be accepted.


Although it is technically compatible, it would make numpy effectively
GPL.  Suggestions for this have been explicitly rejected on these
grounds [1]

[1] https://github.com/numpy/numpy/issues/3485



Yes it would make numpy GPL, but that is not a concern for a lot of 
users. Users for who it is a problem can still use the non-GPL version.
A more interesting debate is whether our binary wheels should then be 
GPL wheels by default or not. Probably not, but that is something that 
should be discussed when its an actual issue.


But to clarify what I said, it would be accepted if the value it 
provides is sufficient compared to the code maintenance it adds. Given 
that pyfftw already exists the value is probably relatively small, but 
personally I'd still be interested in code that allows switching the fft 
backend as that could also allow plugging e.g. gpu based implementations 
(though again this is already covered by other third party modules).

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Todd 
On Thu, Oct 27, 2016 at 10:43 AM, Julian Taylor <
jtaylor.deb...@googlemail.com> wrote:

> On 10/27/2016 04:30 PM, Todd wrote:
>
>> On Thu, Oct 27, 2016 at 4:25 AM, Ralf Gommers > > wrote:
>>
>>
>> On Thu, Oct 27, 2016 at 10:25 AM, Pavlyk, Oleksandr
>> >
>> wrote:
>>
>> Please see responses inline.
>>
>>
>>
>> *From:*NumPy-Discussion
>> [mailto:numpy-discussion-boun...@scipy.org
>> ] *On Behalf Of *Todd
>> *Sent:* Wednesday, October 26, 2016 4:04 PM
>> *To:* Discussion of Numerical Python > >
>> *Subject:* Re: [Numpy-discussion] Intel random number package
>>
>>
>>
>>
>> On Wed, Oct 26, 2016 at 4:30 PM, Pavlyk, Oleksandr
>> >
>> wrote:
>>
>> Another point already raised by Nathaniel is that for
>> numpy's randomness ideally should provide a way to override
>> default algorithm for sampling from a particular
>> distribution.  For example RandomState object that
>> implements PCG may rely on default acceptance-rejection
>> algorithm for sampling from Gamma, while the RandomState
>> object that provides interface to MKL might want to call
>> into MKL directly.
>>
>>
>>
>> The approach that pyfftw uses at least for scipy, which may also
>> work here, is that you can monkey-patch the scipy.fftpack module
>> at runtime, replacing it with pyfftw's drop-in replacement.
>> scipy then proceeds to use pyfftw instead of its built-in
>> fftpack implementation.  Might such an approach work here?
>> Users can either use this alternative randomstate replacement
>> directly, or they can replace numpy's with it at runtime and
>> numpy will then proceed to use the alternative.
>>
>>
>> The only reason that pyfftw uses monkeypatching is that the better
>> approach is not possible due to license constraints with FFTW (it's
>> GPL).
>>
>>
>> Yes, that is exactly why I brought it up.  Better approaches are also
>> not possible with MKL due to license constraints.  It is a very similar
>> situation overall.
>>
>>
> Its not that similar, the better approach is certainly possible with FFTW,
> the GPL is compatible with numpys license. It is only a concern users of
> binary distributions. Nobody provided the code to use fftw yet, but it
> would certainly be accepted.


Although it is technically compatible, it would make numpy effectively
GPL.  Suggestions for this have been explicitly rejected on these grounds
[1]

[1] https://github.com/numpy/numpy/issues/3485
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Julian Taylor 

On 10/27/2016 04:30 PM, Todd wrote:

On Thu, Oct 27, 2016 at 4:25 AM, Ralf Gommers > wrote:


On Thu, Oct 27, 2016 at 10:25 AM, Pavlyk, Oleksandr
> wrote:

Please see responses inline.



*From:*NumPy-Discussion
[mailto:numpy-discussion-boun...@scipy.org
] *On Behalf Of *Todd
*Sent:* Wednesday, October 26, 2016 4:04 PM
*To:* Discussion of Numerical Python >
*Subject:* Re: [Numpy-discussion] Intel random number package




On Wed, Oct 26, 2016 at 4:30 PM, Pavlyk, Oleksandr
>
wrote:

Another point already raised by Nathaniel is that for
numpy's randomness ideally should provide a way to override
default algorithm for sampling from a particular
distribution.  For example RandomState object that
implements PCG may rely on default acceptance-rejection
algorithm for sampling from Gamma, while the RandomState
object that provides interface to MKL might want to call
into MKL directly.



The approach that pyfftw uses at least for scipy, which may also
work here, is that you can monkey-patch the scipy.fftpack module
at runtime, replacing it with pyfftw's drop-in replacement.
scipy then proceeds to use pyfftw instead of its built-in
fftpack implementation.  Might such an approach work here?
Users can either use this alternative randomstate replacement
directly, or they can replace numpy's with it at runtime and
numpy will then proceed to use the alternative.


The only reason that pyfftw uses monkeypatching is that the better
approach is not possible due to license constraints with FFTW (it's
GPL).


Yes, that is exactly why I brought it up.  Better approaches are also
not possible with MKL due to license constraints.  It is a very similar
situation overall.



Its not that similar, the better approach is certainly possible with 
FFTW, the GPL is compatible with numpys license. It is only a concern 
users of binary distributions. Nobody provided the code to use fftw yet, 
but it would certainly be accepted.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Todd 
On Thu, Oct 27, 2016 at 4:25 AM, Ralf Gommers 
wrote:
>
>
> On Thu, Oct 27, 2016 at 10:25 AM, Pavlyk, Oleksandr <
> oleksandr.pav...@intel.com> wrote:
>
>> Please see responses inline.
>>
>>
>>
>> *From:* NumPy-Discussion [mailto:numpy-discussion-boun...@scipy.org] *On
>> Behalf Of *Todd
>> *Sent:* Wednesday, October 26, 2016 4:04 PM
>> *To:* Discussion of Numerical Python 
>> *Subject:* Re: [Numpy-discussion] Intel random number package
>>
>>
>>
>> On Wed, Oct 26, 2016 at 4:30 PM, Pavlyk, Oleksandr <
>> oleksandr.pav...@intel.com> wrote:
>>
>> Another point already raised by Nathaniel is that for numpy's randomness
>> ideally should provide a way to override default algorithm for sampling
>> from a particular distribution.  For example RandomState object that
>> implements PCG may rely on default acceptance-rejection algorithm for
>> sampling from Gamma, while the RandomState object that provides interface
>> to MKL might want to call into MKL directly.
>>
>>
>>
>> The approach that pyfftw uses at least for scipy, which may also work
>> here, is that you can monkey-patch the scipy.fftpack module at runtime,
>> replacing it with pyfftw's drop-in replacement.  scipy then proceeds to use
>> pyfftw instead of its built-in fftpack implementation.  Might such an
>> approach work here?  Users can either use this alternative randomstate
>> replacement directly, or they can replace numpy's with it at runtime and
>> numpy will then proceed to use the alternative.
>>
>
> The only reason that pyfftw uses monkeypatching is that the better
> approach is not possible due to license constraints with FFTW (it's GPL).
>

Yes, that is exactly why I brought it up.  Better approaches are also not
possible with MKL due to license constraints.  It is a very similar
situation overall.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Problem with compiling openacc with f2py

2016-10-27 Thread Vikram Singh 
I am a newbie to f2py so I have been creating simple test cases.
Eventually I want to be able to use openacc subroutine from python. So
here's the test case

module test

 use iso_c_binding, only: sp => C_FLOAT, dp => C_DOUBLE, i8 => C_INT
 use omp_lib
 use openacc

 implicit none

 contains

   subroutine add_acc (a, b, n, c)
  integer(kind=i8), intent(in)  :: n
  real(kind=dp), intent(in)  :: a(n)
  real(kind=dp), intent(in)  :: b(n)
  real(kind=dp), intent(out) :: c(n)

  integer(kind=i8)  :: i

  !$acc kernels
  do i = 1, n
  c(i) = a(i) + b(i)
  end do
  !$acc end kernels

  end subroutine add_acc

  subroutine add_omp (a, b, n, c)
  integer(kind=i8), intent(in)  :: n
  real(kind=dp), intent(in)  :: a(n)
  real(kind=dp), intent(in)  :: b(n)
  real(kind=dp), intent(out) :: c(n)

  integer(kind=i8)  :: i, j

  !$omp parallel do
  do i = 1, n
  c(i) = a(i) + b(i)
  end do
  !$omp end parallel do

  end subroutine add_omp

  subroutine nt (c)
  integer(kind=i8), intent(out) :: c

  c = omp_get_max_threads()

  end subroutine nt

  subroutine mult (a, b, c)
  real(kind=dp), intent(in)  :: a
  real(kind=dp), intent(in)  :: b
  real(kind=dp), intent(out) :: c

  c = a * b

  end subroutine mult

end module test

I compile using

f2py -c -m --f90flags='-fopenacc -foffload=nvptx-none -foffload=-O3
-O3 -fPIC' hello hello.f90 -L/usr/local/cuda/lib64 -lcublas -lcudart
-lgomp

Now, until I add the acc directives everything works fine. But as soon
as I add the acc directives I get this error.

gfortran:f90: /tmp/tmpld6ssow3/src.linux-x86_64-3.5/hello-f2pywrappers2.f90
/home//Experiments/Nvidia/OpenACC/OLCFHack15/gcc6/install/bin/gfortran
-Wall -g -Wall -g -shared
/tmp/tmpld6ssow3/tmp/tmpld6ssow3/src.linux-x86_64-3.5/hellomodule.o
/tmp/tmpld6ssow3/tmp/tmpld6ssow3/src.linux-x86_64-3.5/fortranobject.o
/tmp/tmpld6ssow3/hello.o
/tmp/tmpld6ssow3/tmp/tmpld6ssow3/src.linux-x86_64-3.5/hello-f2pywrappers2.o
-L/usr/local/cuda/lib64 -L/home//usr/local/miniconda/lib -lcublas
-lcudart -lgomp -lpython3.5m -lgfortran -o
./hello.cpython-35m-x86_64-linux-gnu.so
/usr/bin/ld: /tmp/cc2yQ89d.target.o: relocation R_X86_64_32 against
`.rodata' can not be used when making a shared object; recompile with
-fPIC
/tmp/cc2yQ89d.target.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
/usr/bin/ld: /tmp/cc2yQ89d.target.o: relocation R_X86_64_32 against
`.rodata' can not be used when making a shared object; recompile with
-fPIC
/tmp/cc2yQ89d.target.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
error: Command 
"/home//Experiments/Nvidia/OpenACC/OLCFHack15/gcc6/install/bin/gfortran
-Wall -g -Wall -g -shared
/tmp/tmpld6ssow3/tmp/tmpld6ssow3/src.linux-x86_64-3.5/hellomodule.o
/tmp/tmpld6ssow3/tmp/tmpld6ssow3/src.linux-x86_64-3.5/fortranobject.o
/tmp/tmpld6ssow3/hello.o
/tmp/tmpld6ssow3/tmp/tmpld6ssow3/src.linux-x86_64-3.5/hello-f2pywrappers2.o
-L/usr/local/cuda/lib64 -L/home//usr/local/miniconda/lib -lcublas
-lcudart -lgomp -lpython3.5m -lgfortran -o
./hello.cpython-35m-x86_64-linux-gnu.so" failed with exit status 1

I don't get why just putting acc directives should create errors, when
omp does not.

Vikram
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Intel random number package

2016-10-27 Thread Ralf Gommers 
On Thu, Oct 27, 2016 at 10:25 AM, Pavlyk, Oleksandr <
oleksandr.pav...@intel.com> wrote:

> Please see responses inline.
>
>
>
> *From:* NumPy-Discussion [mailto:numpy-discussion-boun...@scipy.org] *On
> Behalf Of *Todd
> *Sent:* Wednesday, October 26, 2016 4:04 PM
> *To:* Discussion of Numerical Python 
> *Subject:* Re: [Numpy-discussion] Intel random number package
>
>
>
> On Wed, Oct 26, 2016 at 4:30 PM, Pavlyk, Oleksandr <
> oleksandr.pav...@intel.com> wrote:
>
> Another point already raised by Nathaniel is that for numpy's randomness
> ideally should provide a way to override default algorithm for sampling
> from a particular distribution.  For example RandomState object that
> implements PCG may rely on default acceptance-rejection algorithm for
> sampling from Gamma, while the RandomState object that provides interface
> to MKL might want to call into MKL directly.
>
>
>
> The approach that pyfftw uses at least for scipy, which may also work
> here, is that you can monkey-patch the scipy.fftpack module at runtime,
> replacing it with pyfftw's drop-in replacement.  scipy then proceeds to use
> pyfftw instead of its built-in fftpack implementation.  Might such an
> approach work here?  Users can either use this alternative randomstate
> replacement directly, or they can replace numpy's with it at runtime and
> numpy will then proceed to use the alternative.
>

The only reason that pyfftw uses monkeypatching is that the better approach
is not possible due to license constraints with FFTW (it's GPL).


> I think the monkey-patching approach will work.
>

It will work, for a while at least, but it's bad design.

We're all on the same page I think that a separate submodule for
random_intel is a no go, but as an explicitly switchable backend for
functions with the same signature it would be fine imho. Of course we don't
have that backend infrastructure today, but it's something we want and have
been discussing anyway.

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion