Re: [Numpy-discussion] Numpy 1.9.1, zeros and alignement

[Pauli Virtanen]

18.11.2014, 21:44, David Cournapeau kirjoitti:
> On Tue, Nov 18, 2014 at 7:05 PM, Julian Taylor <
> jtaylor.deb...@googlemail.com> wrote:
> 
>> 32 bit windows should not provide 16 byte alignment, at least it doesn't
>> for me. That is typically a property of 64 bit OS.
>
>> But that does not explain why normal double is not aligned for you, that
>> only needs 4 bytes on i386 which even 32 bit OS should provide.
> 
> Sorry for the confusion, doubles are aligned, only complex128 are not. But
> I see that on linux 32 bits, this is the same as on windows (zeros output
> not always aligned on "D" dtype), and yet I don't see the issues with f2py
> not being able to

The scipy ticket is here, btw:
https://github.com/scipy/scipy/issues/4168

The second question is whether F2py actually *needs* to check the
dtype-size alignment, or is just something like sizeof(double) enough
for Fortran compilers. Fortran compilers however apparently do generate
code that crashes and burns if there's no alignment also on x86:
https://github.com/scipy/scipy/pull/2698  All this is probably
unspecified, as it's just F77 up out there.

Apparently, everything has worked OK with the old Numpy behavior, or at
least, nobody managed to pinpoint a crash on Win32 because of this? Can
the F2py alignment checks be relaxed? Maybe it is enough to assume the
Fortran compiler is happy with whatever alignment the system malloc()
assures?

If not, the second option is a bit nasty, since I'd believe many people
have f2py code out there with complex inout arrays, and I think no-one
uses special aligned allocators...

-- 
Pauli Virtanen

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.9.1, zeros and alignement

[David Cournapeau]

On Tue, Nov 18, 2014 at 7:05 PM, Julian Taylor <
jtaylor.deb...@googlemail.com> wrote:

> 32 bit windows should not provide 16 byte alignment, at least it doesn't
> for me. That is typically a property of 64 bit OS.
>

> But that does not explain why normal double is not aligned for you, that
> only needs 4 bytes on i386 which even 32 bit OS should provide.
>

Sorry for the confusion, doubles are aligned, only complex128 are not. But
I see that on linux 32 bits, this is the same as on windows (zeros output
not always aligned on "D" dtype), and yet I don't see the issues with f2py
not being able to

I am starting to think I got on the wrong track regarding the original
issue I see with scipy 0.14.x on win 32 bits... I will create a separate
ticket for that.

David
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.9.1, zeros and alignement

[Julian Taylor]

32 bit windows should not provide 16 byte alignment, at least it doesn't
for me. That is typically a property of 64 bit OS.

But that does not explain why normal double is not aligned for you, that
only needs 4 bytes on i386 which even 32 bit OS should provide.

I though I tested scipy on 32 bit linux which has very similar
properties to win32, time to rerun the test.

On 18.11.2014 19:56, David Cournapeau wrote:
> 
> 
> On Tue, Nov 18, 2014 at 6:40 PM, Julian Taylor
> mailto:jtaylor.deb...@googlemail.com>>
> wrote:
> 
> < 1.9 lies about alignment it doesn't actually check for new arrays.
> 
> 
> When I do the following on 1.8.1 with win 32 bits:
> 
> x = np.zeros(12, "D")
> print x.aligned.flags  == (x.__array_interface__["data"][0] % 16 == 0) #
> always true
> print x.aligned.flags # always true
> 
> but on 1.9.1:
> 
> x = np.zeros(12, "D")
> print x.aligned.flags == (x.__array_interface__["data"][0] % 16 == 0) #
> always true
> print x.aligned.flags # not always true
> 
> I wonder why numpy 1.8.1 always returned 16 bytes aligned arrays in this
> case (unlikely to be a coincidence, as I try quite a few times with
> different sizes).
>  
> 
> 
> is the array aligned?
> 
> On 18.11.2014 19:37, David Cournapeau wrote:
> > Additional point: it seems to always return aligned data on 1.8.1 (same
> > platform/compiler/everything).
> >
> > On Tue, Nov 18, 2014 at 6:35 PM, David Cournapeau  
> > >> wrote:
> >
> > It is on windows 32 bits, but I would need to make this work for
> > complex (pair of double) as well.
> >
> > Is this a bug (I  assumed array creation methods would always create
> > aligned arrays for their type) ? Seems like quite a bit of code out
> > there would assume this (scipy itself does for example).
> >
> > (the context is > 100 test failures on scipy 0.14.x on top of numpy
> > 1.9., because f2py intent(inout) fails on work arrays created by
> > zeros, this is a windows-32 only failure).
> >
> > David
> >
> > On Tue, Nov 18, 2014 at 6:26 PM, Julian Taylor
> >  
> >  >> wrote:
> >
> > On 18.11.2014 19:20, David Cournapeau wrote:
> > > Hi,
> > >
> > > I have not followed closely the changes that happen in 1.9.1,
> > but was
> > > surprised by the following:
> > >
> > > x = np.zeros(12, "d")
> > > assert x.flags.aligned # fails
> > >
> > > This is running numpy 1.9.1 built on windows with VS 2008. Is 
> it
> > > expected that zeros may return a non-aligned array ?
> > >
> >
> > what is the real alignment of the array? Are you on 32 bit or 64
> > bit?
> > What is the alignment of doubles in windows (linux its 4 byte on
> > 32 bit
> > 8 byte on 64 bit (% special compiler flags)?
> > print x.__array_interface__["data"]
> >
> > there are problems with complex types but doubles should be
> > aligned even
> > on 32 bit.
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> 
> >
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> >
> >
> >
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org 
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org 
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> 
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.9.1, zeros and alignement

[David Cournapeau]

On Tue, Nov 18, 2014 at 6:40 PM, Julian Taylor <
jtaylor.deb...@googlemail.com> wrote:

> < 1.9 lies about alignment it doesn't actually check for new arrays.
>

When I do the following on 1.8.1 with win 32 bits:

x = np.zeros(12, "D")
print x.aligned.flags  == (x.__array_interface__["data"][0] % 16 == 0) #
always true
print x.aligned.flags # always true

but on 1.9.1:

x = np.zeros(12, "D")
print x.aligned.flags == (x.__array_interface__["data"][0] % 16 == 0) #
always true
print x.aligned.flags # not always true

I wonder why numpy 1.8.1 always returned 16 bytes aligned arrays in this
case (unlikely to be a coincidence, as I try quite a few times with
different sizes).


>
> is the array aligned?
>
> On 18.11.2014 19:37, David Cournapeau wrote:
> > Additional point: it seems to always return aligned data on 1.8.1 (same
> > platform/compiler/everything).
> >
> > On Tue, Nov 18, 2014 at 6:35 PM, David Cournapeau  > > wrote:
> >
> > It is on windows 32 bits, but I would need to make this work for
> > complex (pair of double) as well.
> >
> > Is this a bug (I  assumed array creation methods would always create
> > aligned arrays for their type) ? Seems like quite a bit of code out
> > there would assume this (scipy itself does for example).
> >
> > (the context is > 100 test failures on scipy 0.14.x on top of numpy
> > 1.9., because f2py intent(inout) fails on work arrays created by
> > zeros, this is a windows-32 only failure).
> >
> > David
> >
> > On Tue, Nov 18, 2014 at 6:26 PM, Julian Taylor
> >  > > wrote:
> >
> > On 18.11.2014 19:20, David Cournapeau wrote:
> > > Hi,
> > >
> > > I have not followed closely the changes that happen in 1.9.1,
> > but was
> > > surprised by the following:
> > >
> > > x = np.zeros(12, "d")
> > > assert x.flags.aligned # fails
> > >
> > > This is running numpy 1.9.1 built on windows with VS 2008. Is
> it
> > > expected that zeros may return a non-aligned array ?
> > >
> >
> > what is the real alignment of the array? Are you on 32 bit or 64
> > bit?
> > What is the alignment of doubles in windows (linux its 4 byte on
> > 32 bit
> > 8 byte on 64 bit (% special compiler flags)?
> > print x.__array_interface__["data"]
> >
> > there are problems with complex types but doubles should be
> > aligned even
> > on 32 bit.
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org 
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> >
> >
> >
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.9.1, zeros and alignement

[Julian Taylor]

< 1.9 lies about alignment it doesn't actually check for new arrays.

is the array aligned?

On 18.11.2014 19:37, David Cournapeau wrote:
> Additional point: it seems to always return aligned data on 1.8.1 (same
> platform/compiler/everything).
> 
> On Tue, Nov 18, 2014 at 6:35 PM, David Cournapeau  > wrote:
> 
> It is on windows 32 bits, but I would need to make this work for
> complex (pair of double) as well.
> 
> Is this a bug (I  assumed array creation methods would always create
> aligned arrays for their type) ? Seems like quite a bit of code out
> there would assume this (scipy itself does for example).
> 
> (the context is > 100 test failures on scipy 0.14.x on top of numpy
> 1.9., because f2py intent(inout) fails on work arrays created by
> zeros, this is a windows-32 only failure).
> 
> David
> 
> On Tue, Nov 18, 2014 at 6:26 PM, Julian Taylor
>  > wrote:
> 
> On 18.11.2014 19:20, David Cournapeau wrote:
> > Hi,
> >
> > I have not followed closely the changes that happen in 1.9.1,
> but was
> > surprised by the following:
> >
> > x = np.zeros(12, "d")
> > assert x.flags.aligned # fails
> >
> > This is running numpy 1.9.1 built on windows with VS 2008. Is it
> > expected that zeros may return a non-aligned array ?
> >
> 
> what is the real alignment of the array? Are you on 32 bit or 64
> bit?
> What is the alignment of doubles in windows (linux its 4 byte on
> 32 bit
> 8 byte on 64 bit (% special compiler flags)?
> print x.__array_interface__["data"]
> 
> there are problems with complex types but doubles should be
> aligned even
> on 32 bit.
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org 
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> 
> 
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.9.1, zeros and alignement

[David Cournapeau]

Additional point: it seems to always return aligned data on 1.8.1 (same
platform/compiler/everything).

On Tue, Nov 18, 2014 at 6:35 PM, David Cournapeau 
wrote:

> It is on windows 32 bits, but I would need to make this work for complex
> (pair of double) as well.
>
> Is this a bug (I  assumed array creation methods would always create
> aligned arrays for their type) ? Seems like quite a bit of code out there
> would assume this (scipy itself does for example).
>
> (the context is > 100 test failures on scipy 0.14.x on top of numpy 1.9.,
> because f2py intent(inout) fails on work arrays created by zeros, this is a
> windows-32 only failure).
>
> David
>
> On Tue, Nov 18, 2014 at 6:26 PM, Julian Taylor <
> jtaylor.deb...@googlemail.com> wrote:
>
>> On 18.11.2014 19:20, David Cournapeau wrote:
>> > Hi,
>> >
>> > I have not followed closely the changes that happen in 1.9.1, but was
>> > surprised by the following:
>> >
>> > x = np.zeros(12, "d")
>> > assert x.flags.aligned # fails
>> >
>> > This is running numpy 1.9.1 built on windows with VS 2008. Is it
>> > expected that zeros may return a non-aligned array ?
>> >
>>
>> what is the real alignment of the array? Are you on 32 bit or 64 bit?
>> What is the alignment of doubles in windows (linux its 4 byte on 32 bit
>> 8 byte on 64 bit (% special compiler flags)?
>> print x.__array_interface__["data"]
>>
>> there are problems with complex types but doubles should be aligned even
>> on 32 bit.
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.9.1, zeros and alignement

[David Cournapeau]

It is on windows 32 bits, but I would need to make this work for complex
(pair of double) as well.

Is this a bug (I  assumed array creation methods would always create
aligned arrays for their type) ? Seems like quite a bit of code out there
would assume this (scipy itself does for example).

(the context is > 100 test failures on scipy 0.14.x on top of numpy 1.9.,
because f2py intent(inout) fails on work arrays created by zeros, this is a
windows-32 only failure).

David

On Tue, Nov 18, 2014 at 6:26 PM, Julian Taylor <
jtaylor.deb...@googlemail.com> wrote:

> On 18.11.2014 19:20, David Cournapeau wrote:
> > Hi,
> >
> > I have not followed closely the changes that happen in 1.9.1, but was
> > surprised by the following:
> >
> > x = np.zeros(12, "d")
> > assert x.flags.aligned # fails
> >
> > This is running numpy 1.9.1 built on windows with VS 2008. Is it
> > expected that zeros may return a non-aligned array ?
> >
>
> what is the real alignment of the array? Are you on 32 bit or 64 bit?
> What is the alignment of doubles in windows (linux its 4 byte on 32 bit
> 8 byte on 64 bit (% special compiler flags)?
> print x.__array_interface__["data"]
>
> there are problems with complex types but doubles should be aligned even
> on 32 bit.
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.9.1, zeros and alignement

[Julian Taylor]

On 18.11.2014 19:20, David Cournapeau wrote:
> Hi,
> 
> I have not followed closely the changes that happen in 1.9.1, but was
> surprised by the following:
> 
> x = np.zeros(12, "d")
> assert x.flags.aligned # fails
> 
> This is running numpy 1.9.1 built on windows with VS 2008. Is it
> expected that zeros may return a non-aligned array ?
> 

what is the real alignment of the array? Are you on 32 bit or 64 bit?
What is the alignment of doubles in windows (linux its 4 byte on 32 bit
8 byte on 64 bit (% special compiler flags)?
print x.__array_interface__["data"]

there are problems with complex types but doubles should be aligned even
on 32 bit.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.9.1, zeros and alignement

[Charles R Harris]

On Tue, Nov 18, 2014 at 11:20 AM, David Cournapeau 
wrote:

> Hi,
>
> I have not followed closely the changes that happen in 1.9.1, but was
> surprised by the following:
>
> x = np.zeros(12, "d")
> assert x.flags.aligned # fails
>
> This is running numpy 1.9.1 built on windows with VS 2008. Is it expected
> that zeros may return a non-aligned array ?
>

The alignment requirements may have become stricter, Julian would know. Is
this on 32 or 64 Windows?

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Numpy 1.9.1, zeros and alignement

[David Cournapeau]

Hi,

I have not followed closely the changes that happen in 1.9.1, but was
surprised by the following:

x = np.zeros(12, "d")
assert x.flags.aligned # fails

This is running numpy 1.9.1 built on windows with VS 2008. Is it expected
that zeros may return a non-aligned array ?

David
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion