Re: [Numpy-discussion] ENH: `scale` parameter for `sinc`

[Nathaniel Smith]

On Tue, Feb 23, 2016 at 3:20 PM, Joseph Fox-Rabinovitz
 wrote:
> P.S. I would like to turn `sinc` into a `ufunc` at some point if the
> community approves. It would make the computation much cleaner (e.g.,
> in-place `where`) and faster. It would also complement the existing
> trig functions nicely. The only question I have is whether or not it
> is possible to pass in optional parameters to ufuncs beyond the ones
> listed in 
> http://docs.scipy.org/doc/numpy-1.10.0/reference/ufuncs.html#optional-keyword-arguments

Right now it isn't possible, no. There are a lot of general
improvements we can/should make to ufuncs, and this is one of them...

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] ENH: `scale` parameter for `sinc`

[Joseph Fox-Rabinovitz]

I have created PR #7322 (https://github.com/numpy/numpy/pull/7322) to
add a scale parameter to `sinc`. What this allows is to compute `sinc`
as `sin(x)/x` or really `sin(n*x)/(n*x)` for arbitrary `n` instead of
just `sin(pi*x)/(pi*x)` as is being done now. The parameter accepts
two string arguments in addition to the actual scale value:
'normalized' and 'unnormalized'. 'normalized' is the default since
that is the existing functionality. 'unnormalized' is equivalent to a
`scale` of 1.0. The parameter also supports broadcasting against the
input array.

Regards,

-Joe

P.S. I would like to turn `sinc` into a `ufunc` at some point if the
community approves. It would make the computation much cleaner (e.g.,
in-place `where`) and faster. It would also complement the existing
trig functions nicely. The only question I have is whether or not it
is possible to pass in optional parameters to ufuncs beyond the ones
listed in 
http://docs.scipy.org/doc/numpy-1.10.0/reference/ufuncs.html#optional-keyword-arguments
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.11.0rc1 released.

[Charles R Harris]

Christoph, any chance you can test https://github.com/numpy/numpy/pull/7324
before it gets merged (or not).



Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.11.0rc1 released.

[Christoph Gohlke]

On 2/23/2016 1:05 PM, Pauli Virtanen wrote:

23.02.2016, 22:40, Charles R Harris kirjoitti:
[clip]

On all 32-bit platforms:


ERROR: test_zeros_big (test_multiarray.TestCreation)

Traceback (most recent call last):
   File "X:\Python27\lib\site-packages\numpy\core\tests\test_multiarray.py",
line 594, in test_zeros_big
 d = np.zeros((30 * 1024**2,), dtype=dt)
MemoryError

I would be much obliged if someone else could demonstrate it.


Memory fragmentation in the 2GB address space available? If dt==float64,
that requires 250MB contiguous.



Before creating the dtype('D') test array, the largest contiguous block 
available to the 32 bit Python process on my system is ~830 MB. The 
creation of this array succeeds. However, the creation of the next 
dtype('G') test array fails because the previous array is still in 
memory and the largest contiguous block available is only ~318 MB. 
Deleting the test arrays after usage via del(d) fixes this problem 
. Another fix could be to 
change the order of data types tested.


Christoph
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.11.0rc1 released.

[Charles R Harris]

On Tue, Feb 23, 2016 at 1:58 PM, Julian Taylor <
jtaylor.deb...@googlemail.com> wrote:

> that test needs about 500Mb of memory on windows as it doesn't have
> sparse allocations like most *nixes.
> It used to fail for me during release testing when I only gave the
> windows VM 1GB of ram.
> If its a problem for CI we could disable it on windows, or at least skip
> the complex double case.
>

It's not a problem on CI, just for Christoph. I asked him what memory
resources the test had available but haven't heard back. AFAICT, nothing
associated with the test has changed for this release. The options are
probably to 1) ignore the failure, or 2) disable the test on 32 bits.



Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to check for memory leaks?

[Sebastian Berg]

On Di, 2016-02-23 at 12:36 -0700, Charles R Harris wrote:
> Hi All,
> 
> I'm suspecting a possible memory leak in 1.11.x, what is the best way
> to check for that?
> 

Would like to learn better methods, but I tried valgrind with trace
origins and full leak check, just thinking maybe it shows something.
Unfortunately, I got the error below midway, I ran it before
successfully (with only minor obvious leaks due to things like module
wide strings) I think. My guess is, the error does not say much at all,
but I have no clue :) (running without track-origins now, maybe it
helps).

- Sebastian

Error:

VEX temporary storage exhausted.
Pool = TEMP,  start 0x38f91668 curr 0x39456190 end 0x394561a7 (size
500)

vex: the `impossible' happened:
   VEX temporary storage exhausted.



> Chuck
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion

signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.11.0rc1 released.

[Pauli Virtanen]

23.02.2016, 22:40, Charles R Harris kirjoitti:
[clip]
> On all 32-bit platforms:
> 
> 
> ERROR: test_zeros_big (test_multiarray.TestCreation)
> 
> Traceback (most recent call last):
>   File "X:\Python27\lib\site-packages\numpy\core\tests\test_multiarray.py",
> line 594, in test_zeros_big
> d = np.zeros((30 * 1024**2,), dtype=dt)
> MemoryError
> 
> I would be much obliged if someone else could demonstrate it.

Memory fragmentation in the 2GB address space available? If dt==float64,
that requires 250MB contiguous.

-- 
Pauli Virtanen

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.11.0rc1 released.

[Julian Taylor]

that test needs about 500Mb of memory on windows as it doesn't have
sparse allocations like most *nixes.
It used to fail for me during release testing when I only gave the
windows VM 1GB of ram.
If its a problem for CI we could disable it on windows, or at least skip
the complex double case.

On 23.02.2016 21:40, Charles R Harris wrote:
> Christoph reports the following problem that I am unable to reproduce on
> appveyor or find reported elsewhere.
> 
> On all 32-bit platforms:
> 
> 
> ERROR: test_zeros_big (test_multiarray.TestCreation)
> 
> Traceback (most recent call last):
>   File
> "X:\Python27\lib\site-packages\numpy\core\tests\test_multiarray.py",
> line 594, in test_zeros_big
> d = np.zeros((30 * 1024**2,), dtype=dt)
> MemoryError
> 
> I would be much obliged if someone else could demonstrate it.
> 
> 
> 
> Chuck
> 
> 
> 
> 
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
> 

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] reshaping empty array bug?

[Benjamin Root]

On Tue, Feb 23, 2016 at 3:30 PM, Nathaniel Smith  wrote:

> What should this do?
>
> np.zeros((12, 0)).reshape((10, -1, 2))
>


It should error out, I already covered that. 12 != 20.

Ben Root
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.11.0rc1 released.

[Charles R Harris]

Christoph reports the following problem that I am unable to reproduce on
appveyor or find reported elsewhere.

On all 32-bit platforms:


ERROR: test_zeros_big (test_multiarray.TestCreation)

Traceback (most recent call last):
  File "X:\Python27\lib\site-packages\numpy\core\tests\test_multiarray.py",
line 594, in test_zeros_big
d = np.zeros((30 * 1024**2,), dtype=dt)
MemoryError

I would be much obliged if someone else could demonstrate it.



Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] reshaping empty array bug?

[Nathaniel Smith]

On Tue, Feb 23, 2016 at 12:23 PM, Benjamin Root  wrote:
>
> On Tue, Feb 23, 2016 at 3:14 PM, Nathaniel Smith  wrote:
>>
>> Sure, it's totally ambiguous. These are all legal:
>
>
>
> I would argue that except for the first reshape, all of those should be an
> error, and that the current algorithm is buggy.

Reshape doesn't care about axes at all; all it cares about is that the
number of elements stay the same. E.g. this is also totally legal:

np.zeros((12, 5)).reshape((10, 3, 2))

And so are the equivalents

np.zeros((12, 5)).reshape((-1, 3, 2))
np.zeros((12, 5)).reshape((10, -1, 2))
np.zeros((12, 5)).reshape((10, 3, -1))

> This isn't a heuristic. It isn't guessing. It is making the semantics
> consistent. The fact that I can do:
> a.shape = (-1, 5, 64)
> or
> a.shape = (0, 5, 64)
>
> but not
> a.shape = (0, 5, -1)
>
> is totally inconsistent.

It's certainly annoying and unpleasant, but it follows inevitably from
the most natural way of defining the -1 semantics, so I'm not sure I'd
say "inconsistent" :-)

What should this do?

np.zeros((12, 0)).reshape((10, -1, 2))

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] reshaping empty array bug?

[Benjamin Root]

On Tue, Feb 23, 2016 at 3:14 PM, Nathaniel Smith  wrote:

> Sure, it's totally ambiguous. These are all legal:



I would argue that except for the first reshape, all of those should be an
error, and that the current algorithm is buggy.

This isn't a heuristic. It isn't guessing. It is making the semantics
consistent. The fact that I can do:
a.shape = (-1, 5, 64)
or
a.shape = (0, 5, 64)

but not
a.shape = (0, 5, -1)

is totally inconsistent.

Ben Root
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] reshaping empty array bug?

[Nathaniel Smith]

On Tue, Feb 23, 2016 at 8:45 AM, Benjamin Root  wrote:
> but, it isn't really ambiguous, is it? The -1 can only refer to a single
> dimension, and if you ignore the zeros in the original and new shape, the -1
> is easily solvable, right?

Sure, it's totally ambiguous. These are all legal:

In [1]: a = np.zeros((0, 5, 64))

In [2]: a.shape = (0, 5 * 64)

In [3]: a.shape = (0, 5 * 65)

In [4]: a.shape = (0, 5, 102)

In [5]: a.shape = (0, 102, 64)

Generally, the -1 gets replaced by prod(old_shape) //
prod(specified_entries_in_new_shape). If the specified new shape has a
0 in it, then this is a divide-by-zero. In this case it happens
because it's the solution to the equation
  prod((0, 5, 64)) == prod((0, 5, x))
for which there is no unique solution for 'x'.

Your proposed solution feels very heuristic-y to me, and heuristics
make me very nervous :-/

If what you really want to say is "flatten axes 1 and 2 together",
then maybe there should be some API that lets you directly specify
*that*? As a bonus you might be able to avoid awkward tuple
manipulations to compute the new shape.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] reshaping empty array bug?

[Sebastian Berg]

On Di, 2016-02-23 at 21:06 +0100, Sebastian Berg wrote:
> On Di, 2016-02-23 at 14:57 -0500, Benjamin Root wrote:
> > I'd be more than happy to write up the patch. I don't think it
> > would
> > be quite like make zeros be ones, but it would be along those
> > lines.
> > One case I need to wrap my head around is to make sure that a 0
> > would
> > happen if the following was true:
> > 
> > > > > a = np.ones((0, 5*64))
> > > > > a.shape = (-1, 5, 64)
> > 
> > EDIT: Just tried the above, and it works as expected (zero in the
> > first dim)!
> > 
> > Just tried out a couple of other combos:
> > > > > a.shape = (-1,)
> > > > > a.shape
> > (0,)
> > > > > a.shape = (-1, 5, 64)
> > > > > a.shape
> > (0, 5, 64)
> > 
> 
> Seems right to me on first sight :). (I don't like shape assignments
> though, who cares for one extra view). Well, maybe 1 instead of 0
> (ignore 0s), but if the result for -1 is to use 1 and the shape is 0
> convert the 1 back to 0. But it is starting to sound a bit tricky,
> though I think it might be straight forward (i.e. no real traps and
> when it works it always is what you expect).
> The main point is, whether you can design cases where the conversion
> back to 0 hides bugs by not failing when it should. And whether that
> would be a tradeoff we are willing to accept.
> 

Another thought. Maybe you can figure out the -1 correctly, if there is
no *other* 0 involved. If there is any other 0, I could imagine
problems.

> - Sebastian
> 
> 
> > 
> > This is looking more and more like a bug to me.
> > 
> > Ben Root
> > 
> > 
> > On Tue, Feb 23, 2016 at 1:58 PM, Sebastian Berg <
> > sebast...@sipsolutions.net> wrote:
> > > On Di, 2016-02-23 at 11:45 -0500, Benjamin Root wrote:
> > > > but, it isn't really ambiguous, is it? The -1 can only refer to
> > > > a
> > > > single dimension, and if you ignore the zeros in the original
> > > > and
> > > new
> > > > shape, the -1 is easily solvable, right?
> > > 
> > > I think if there is a simple logic (like using 1 for all zeros in
> > > both
> > > input and output shape for the -1 calculation), maybe we could do
> > > it. I
> > > would like someone to think about it carefully that it would not
> > > also
> > > allow some unexpected generalizations. And at least I am getting
> > > a
> > > BrainOutOfResourcesError right now trying to figure that out :).
> > > 
> > > - Sebastian
> > > 
> > > 
> > > > Ben Root
> > > > 
> > > > On Tue, Feb 23, 2016 at 11:41 AM, Warren Weckesser <
> > > > warren.weckes...@gmail.com> wrote:
> > > > > 
> > > > > 
> > > > > On Tue, Feb 23, 2016 at 11:32 AM, Benjamin Root <
> > > > > ben.v.r...@gmail.com> wrote:
> > > > > > Not exactly sure if this should be a bug or not. This came
> > > > > > up
> > > in
> > > > > > a fairly general function of mine to process satellite
> > > > > > data.
> > > > > > Unexpectedly, one of the satellite files had no scans in
> > > > > > it,
> > > > > > triggering an exception when I tried to reshape the data
> > > > > > from
> > > it.
> > > > > > 
> > > > > > > > > import numpy as np
> > > > > > > > > a = np.zeros((0, 5*64))
> > > > > > > > > a.shape
> > > > > > (0, 320)
> > > > > > > > > a.shape = (0, 5, 64)
> > > > > > > > > a.shape
> > > > > > (0, 5, 64)
> > > > > > > > > a.shape = (0, 5*64)
> > > > > > > > > a.shape = (0, 5, -1)
> > > > > > Traceback (most recent call last):
> > > > > >   File "", line 1, in 
> > > > > > ValueError: total size of new array must be unchanged
> > > > > > 
> > > > > > So, if I know all of the dimensions, I can reshape just
> > > > > > fine.
> > > But
> > > > > > if I wanted to use the nifty -1 semantic, it completely
> > > > > > falls
> > > > > > apart. I can see arguments going either way for whether
> > > > > > this
> > > is a
> > > > > > bug or not.
> > > > > > 
> > > > > 
> > > > > When you try `a.shape = (0, 5, -1)`, the size of the third
> > > > > dimension is ambiguous.  From the Zen of Python:  "In the
> > > > > face
> > > of
> > > > > ambiguity, refuse the temptation to guess."
> > > > > 
> > > > > Warren
> > > > > 
> > > > > 
> > > > > 
> > > > > > 
> > > > > > Thoughts?
> > > > > > 
> > > > > > Ben Root
> > > > > > 
> > > > > > ___
> > > > > > NumPy-Discussion mailing list
> > > > > > NumPy-Discussion@scipy.org
> > > > > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > > > > > 
> > > > > 
> > > > > ___
> > > > > NumPy-Discussion mailing list
> > > > > NumPy-Discussion@scipy.org
> > > > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > > > > 
> > > > ___
> > > > NumPy-Discussion mailing list
> > > > NumPy-Discussion@scipy.org
> > > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > > 
> > > ___
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion@scipy.org
> > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > > 
> > _

Re: [Numpy-discussion] reshaping empty array bug?

[Sebastian Berg]

On Di, 2016-02-23 at 14:57 -0500, Benjamin Root wrote:
> I'd be more than happy to write up the patch. I don't think it would
> be quite like make zeros be ones, but it would be along those lines.
> One case I need to wrap my head around is to make sure that a 0 would
> happen if the following was true:
> 
> >>> a = np.ones((0, 5*64))
> >>> a.shape = (-1, 5, 64)
> 
> EDIT: Just tried the above, and it works as expected (zero in the
> first dim)!
> 
> Just tried out a couple of other combos:
> >>> a.shape = (-1,)
> >>> a.shape
> (0,)
> >>> a.shape = (-1, 5, 64)
> >>> a.shape
> (0, 5, 64)
> 

Seems right to me on first sight :). (I don't like shape assignments
though, who cares for one extra view). Well, maybe 1 instead of 0
(ignore 0s), but if the result for -1 is to use 1 and the shape is 0
convert the 1 back to 0. But it is starting to sound a bit tricky,
though I think it might be straight forward (i.e. no real traps and
when it works it always is what you expect).
The main point is, whether you can design cases where the conversion
back to 0 hides bugs by not failing when it should. And whether that
would be a tradeoff we are willing to accept.

- Sebastian


> 
> This is looking more and more like a bug to me.
> 
> Ben Root
> 
> 
> On Tue, Feb 23, 2016 at 1:58 PM, Sebastian Berg <
> sebast...@sipsolutions.net> wrote:
> > On Di, 2016-02-23 at 11:45 -0500, Benjamin Root wrote:
> > > but, it isn't really ambiguous, is it? The -1 can only refer to a
> > > single dimension, and if you ignore the zeros in the original and
> > new
> > > shape, the -1 is easily solvable, right?
> > 
> > I think if there is a simple logic (like using 1 for all zeros in
> > both
> > input and output shape for the -1 calculation), maybe we could do
> > it. I
> > would like someone to think about it carefully that it would not
> > also
> > allow some unexpected generalizations. And at least I am getting a
> > BrainOutOfResourcesError right now trying to figure that out :).
> > 
> > - Sebastian
> > 
> > 
> > > Ben Root
> > >
> > > On Tue, Feb 23, 2016 at 11:41 AM, Warren Weckesser <
> > > warren.weckes...@gmail.com> wrote:
> > > >
> > > >
> > > > On Tue, Feb 23, 2016 at 11:32 AM, Benjamin Root <
> > > > ben.v.r...@gmail.com> wrote:
> > > > > Not exactly sure if this should be a bug or not. This came up
> > in
> > > > > a fairly general function of mine to process satellite data.
> > > > > Unexpectedly, one of the satellite files had no scans in it,
> > > > > triggering an exception when I tried to reshape the data from
> > it.
> > > > >
> > > > > >>> import numpy as np
> > > > > >>> a = np.zeros((0, 5*64))
> > > > > >>> a.shape
> > > > > (0, 320)
> > > > > >>> a.shape = (0, 5, 64)
> > > > > >>> a.shape
> > > > > (0, 5, 64)
> > > > > >>> a.shape = (0, 5*64)
> > > > > >>> a.shape = (0, 5, -1)
> > > > > Traceback (most recent call last):
> > > > >   File "", line 1, in 
> > > > > ValueError: total size of new array must be unchanged
> > > > >
> > > > > So, if I know all of the dimensions, I can reshape just fine.
> > But
> > > > > if I wanted to use the nifty -1 semantic, it completely falls
> > > > > apart. I can see arguments going either way for whether this
> > is a
> > > > > bug or not.
> > > > >
> > > >
> > > > When you try `a.shape = (0, 5, -1)`, the size of the third
> > > > dimension is ambiguous.  From the Zen of Python:  "In the face
> > of
> > > > ambiguity, refuse the temptation to guess."
> > > >
> > > > Warren
> > > >
> > > >
> > > >
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Ben Root
> > > > >
> > > > > ___
> > > > > NumPy-Discussion mailing list
> > > > > NumPy-Discussion@scipy.org
> > > > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > > > >
> > > >
> > > > ___
> > > > NumPy-Discussion mailing list
> > > > NumPy-Discussion@scipy.org
> > > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > > >
> > > ___
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion@scipy.org
> > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > 
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion

signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] reshaping empty array bug?

[Benjamin Root]

I'd be more than happy to write up the patch. I don't think it would be
quite like make zeros be ones, but it would be along those lines. One case
I need to wrap my head around is to make sure that a 0 would happen if the
following was true:

>>> a = np.ones((0, 5*64))
>>> a.shape = (-1, 5, 64)

EDIT: Just tried the above, and it works as expected (zero in the first
dim)!

Just tried out a couple of other combos:
>>> a.shape = (-1,)
>>> a.shape
(0,)
>>> a.shape = (-1, 5, 64)
>>> a.shape
(0, 5, 64)


This is looking more and more like a bug to me.

Ben Root


On Tue, Feb 23, 2016 at 1:58 PM, Sebastian Berg 
wrote:

> On Di, 2016-02-23 at 11:45 -0500, Benjamin Root wrote:
> > but, it isn't really ambiguous, is it? The -1 can only refer to a
> > single dimension, and if you ignore the zeros in the original and new
> > shape, the -1 is easily solvable, right?
>
> I think if there is a simple logic (like using 1 for all zeros in both
> input and output shape for the -1 calculation), maybe we could do it. I
> would like someone to think about it carefully that it would not also
> allow some unexpected generalizations. And at least I am getting a
> BrainOutOfResourcesError right now trying to figure that out :).
>
> - Sebastian
>
>
> > Ben Root
> >
> > On Tue, Feb 23, 2016 at 11:41 AM, Warren Weckesser <
> > warren.weckes...@gmail.com> wrote:
> > >
> > >
> > > On Tue, Feb 23, 2016 at 11:32 AM, Benjamin Root <
> > > ben.v.r...@gmail.com> wrote:
> > > > Not exactly sure if this should be a bug or not. This came up in
> > > > a fairly general function of mine to process satellite data.
> > > > Unexpectedly, one of the satellite files had no scans in it,
> > > > triggering an exception when I tried to reshape the data from it.
> > > >
> > > > >>> import numpy as np
> > > > >>> a = np.zeros((0, 5*64))
> > > > >>> a.shape
> > > > (0, 320)
> > > > >>> a.shape = (0, 5, 64)
> > > > >>> a.shape
> > > > (0, 5, 64)
> > > > >>> a.shape = (0, 5*64)
> > > > >>> a.shape = (0, 5, -1)
> > > > Traceback (most recent call last):
> > > >   File "", line 1, in 
> > > > ValueError: total size of new array must be unchanged
> > > >
> > > > So, if I know all of the dimensions, I can reshape just fine. But
> > > > if I wanted to use the nifty -1 semantic, it completely falls
> > > > apart. I can see arguments going either way for whether this is a
> > > > bug or not.
> > > >
> > >
> > > When you try `a.shape = (0, 5, -1)`, the size of the third
> > > dimension is ambiguous.  From the Zen of Python:  "In the face of
> > > ambiguity, refuse the temptation to guess."
> > >
> > > Warren
> > >
> > >
> > >
> > > >
> > > > Thoughts?
> > > >
> > > > Ben Root
> > > >
> > > > ___
> > > > NumPy-Discussion mailing list
> > > > NumPy-Discussion@scipy.org
> > > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > > >
> > >
> > > ___
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion@scipy.org
> > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > >
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to check for memory leaks?

[Antoine Pitrou]

On Tue, 23 Feb 2016 12:36:00 -0700
Charles R Harris  wrote:
> Hi All,
> 
> I'm suspecting a possible memory leak in 1.11.x, what is the best way to
> check for that?

If that is due to a reference leak, you can use sys.getrefcount() or
weakref.ref().

Otherwise you may want to change Numpy to go through PyMem_RawMalloc /
PyMem_RawCalloc / PyMem_RawRealloc / PyMem_RawFree on recent Pythons,
so as to have Numpy-allocated memory accounted by the tracemalloc
module.

(https://github.com/numpy/numpy/pull/5470 may make it more
palatable ;-))

Regards

Antoine.


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] How to check for memory leaks?

[Charles R Harris]

Hi All,

I'm suspecting a possible memory leak in 1.11.x, what is the best way to
check for that?

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] reshaping empty array bug?

[Sebastian Berg]

On Di, 2016-02-23 at 11:45 -0500, Benjamin Root wrote:
> but, it isn't really ambiguous, is it? The -1 can only refer to a
> single dimension, and if you ignore the zeros in the original and new
> shape, the -1 is easily solvable, right?

I think if there is a simple logic (like using 1 for all zeros in both
input and output shape for the -1 calculation), maybe we could do it. I
would like someone to think about it carefully that it would not also
allow some unexpected generalizations. And at least I am getting a
BrainOutOfResourcesError right now trying to figure that out :).

- Sebastian


> Ben Root
> 
> On Tue, Feb 23, 2016 at 11:41 AM, Warren Weckesser <
> warren.weckes...@gmail.com> wrote:
> > 
> > 
> > On Tue, Feb 23, 2016 at 11:32 AM, Benjamin Root <
> > ben.v.r...@gmail.com> wrote:
> > > Not exactly sure if this should be a bug or not. This came up in
> > > a fairly general function of mine to process satellite data.
> > > Unexpectedly, one of the satellite files had no scans in it,
> > > triggering an exception when I tried to reshape the data from it.
> > > 
> > > >>> import numpy as np
> > > >>> a = np.zeros((0, 5*64))
> > > >>> a.shape
> > > (0, 320)
> > > >>> a.shape = (0, 5, 64)
> > > >>> a.shape
> > > (0, 5, 64)
> > > >>> a.shape = (0, 5*64)
> > > >>> a.shape = (0, 5, -1)
> > > Traceback (most recent call last):
> > >   File "", line 1, in 
> > > ValueError: total size of new array must be unchanged
> > > 
> > > So, if I know all of the dimensions, I can reshape just fine. But
> > > if I wanted to use the nifty -1 semantic, it completely falls
> > > apart. I can see arguments going either way for whether this is a
> > > bug or not.
> > > 
> > 
> > When you try `a.shape = (0, 5, -1)`, the size of the third
> > dimension is ambiguous.  From the Zen of Python:  "In the face of
> > ambiguity, refuse the temptation to guess."
> > 
> > Warren
> > 
> > 
> > 
> > > 
> > > Thoughts?
> > > 
> > > Ben Root
> > > 
> > > ___
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion@scipy.org
> > > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > > 
> > 
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> > 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion

signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] reshaping empty array bug?

[Benjamin Root]

but, it isn't really ambiguous, is it? The -1 can only refer to a single
dimension, and if you ignore the zeros in the original and new shape, the
-1 is easily solvable, right?

Ben Root

On Tue, Feb 23, 2016 at 11:41 AM, Warren Weckesser <
warren.weckes...@gmail.com> wrote:

>
>
> On Tue, Feb 23, 2016 at 11:32 AM, Benjamin Root 
> wrote:
>
>> Not exactly sure if this should be a bug or not. This came up in a fairly
>> general function of mine to process satellite data. Unexpectedly, one of
>> the satellite files had no scans in it, triggering an exception when I
>> tried to reshape the data from it.
>>
>> >>> import numpy as np
>> >>> a = np.zeros((0, 5*64))
>> >>> a.shape
>> (0, 320)
>> >>> a.shape = (0, 5, 64)
>> >>> a.shape
>> (0, 5, 64)
>> >>> a.shape = (0, 5*64)
>> >>> a.shape = (0, 5, -1)
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> ValueError: total size of new array must be unchanged
>>
>> So, if I know all of the dimensions, I can reshape just fine. But if I
>> wanted to use the nifty -1 semantic, it completely falls apart. I can see
>> arguments going either way for whether this is a bug or not.
>>
>
>
> When you try `a.shape = (0, 5, -1)`, the size of the third dimension is
> ambiguous.  From the Zen of Python:  "In the face of ambiguity, refuse the
> temptation to guess."
>
> Warren
>
>
>
>
>> Thoughts?
>>
>> Ben Root
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] reshaping empty array bug?

[Warren Weckesser]

On Tue, Feb 23, 2016 at 11:32 AM, Benjamin Root 
wrote:

> Not exactly sure if this should be a bug or not. This came up in a fairly
> general function of mine to process satellite data. Unexpectedly, one of
> the satellite files had no scans in it, triggering an exception when I
> tried to reshape the data from it.
>
> >>> import numpy as np
> >>> a = np.zeros((0, 5*64))
> >>> a.shape
> (0, 320)
> >>> a.shape = (0, 5, 64)
> >>> a.shape
> (0, 5, 64)
> >>> a.shape = (0, 5*64)
> >>> a.shape = (0, 5, -1)
> Traceback (most recent call last):
>   File "", line 1, in 
> ValueError: total size of new array must be unchanged
>
> So, if I know all of the dimensions, I can reshape just fine. But if I
> wanted to use the nifty -1 semantic, it completely falls apart. I can see
> arguments going either way for whether this is a bug or not.
>


When you try `a.shape = (0, 5, -1)`, the size of the third dimension is
ambiguous.  From the Zen of Python:  "In the face of ambiguity, refuse the
temptation to guess."

Warren




> Thoughts?
>
> Ben Root
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] reshaping empty array bug?

[Benjamin Root]

Not exactly sure if this should be a bug or not. This came up in a fairly
general function of mine to process satellite data. Unexpectedly, one of
the satellite files had no scans in it, triggering an exception when I
tried to reshape the data from it.

>>> import numpy as np
>>> a = np.zeros((0, 5*64))
>>> a.shape
(0, 320)
>>> a.shape = (0, 5, 64)
>>> a.shape
(0, 5, 64)
>>> a.shape = (0, 5*64)
>>> a.shape = (0, 5, -1)
Traceback (most recent call last):
  File "", line 1, in 
ValueError: total size of new array must be unchanged

So, if I know all of the dimensions, I can reshape just fine. But if I
wanted to use the nifty -1 semantic, it completely falls apart. I can see
arguments going either way for whether this is a bug or not.

Thoughts?

Ben Root
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.11.0rc1 released.

[Pauli Virtanen]

23.02.2016, 03:47, Charles R Harris kirjoitti:
> I'm delighted to announce the release of Numpy 1.11.0rc1. Hopefully the
> issues discovered in 1.11.0b3 have been dealt with and this release can go
> on to become the official release. Source files and documentation can be
> found on Sourceforge
> , while
> source files and OS X wheels for Python 2.7, 3.3, 3.4, and 3.5 can be
> installed from Pypi. Please test thoroughly.

FWIW https://travis-ci.org/pv/testrig/builds/108384173

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.11.0rc1 released.

[Charles R Harris]

On Mon, Feb 22, 2016 at 6:47 PM, Charles R Harris  wrote:

> Hi All,
>
> I'm delighted to announce the release of Numpy 1.11.0rc1. Hopefully the
> issues discovered in 1.11.0b3 have been dealt with and this release can go
> on to become the official release. Source files and documentation can be
> found on Sourceforge
> , while
> source files and OS X wheels for Python 2.7, 3.3, 3.4, and 3.5 can be
> installed from Pypi. Please test thoroughly.
>

Issues reported by Christoph at https://github.com/numpy/numpy/issues/7316.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion