[issue35845] Can't read a F-contiguous memoryview in physical order

[jakirkham]

jakirkham  added the comment:

Sorry if I'm just misunderstanding the discussion here. Would it make sense to 
have an `order` keyword argument to `cast` as well? This seems useful when 
interpreting a flatten F-order `bytes` object (say on the receiving end of a 
transmission).

--
nosy: +jakirkham

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Stefan Krah]

Stefan Krah  added the comment:


New changeset d08ea70464cb8a1f86134dcb4a5c2eac1a02bf1a by Stefan Krah in branch 
'master':
bpo-35845: Add order={'C', 'F', 'A'} parameter to memoryview.tobytes(). (#11730)
https://github.com/python/cpython/commit/d08ea70464cb8a1f86134dcb4a5c2eac1a02bf1a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> So we'd need to restrict to contiguous views anyway, which makes
the method less appealing (IOW, it doesn't offer more than an
augmented memoryview.cast()).

Yes, it would probably be a simpler way of writing `.cast('B', shape=(...), 
order='A')`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Stefan Krah]

Stefan Krah  added the comment:

Yes, following NumPy looks like the sanest option for tobytes(), so I
went ahead and implemented that signature.

memory.raw() is of course complicated by the fact that things like
m[::-1] move buf.ptr to the end of the buffer.

So we'd need to restrict to contiguous views anyway, which makes
the method less appealing (IOW, it doesn't offer more than an
augmented memoryview.cast()).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Stefan Krah]

Change by Stefan Krah :


--
keywords: +patch
pull_requests: +11620
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Stefan Krah]

Change by Stefan Krah :


--
keywords: +patch, patch
pull_requests: +11620, 11621
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Stefan Krah]

Change by Stefan Krah :


--
keywords: +patch, patch, patch
pull_requests: +11620, 11621, 11622
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Sorry, my fingers slipped.  Let me try again:

As for tobytes(), if we want to follow NumPy, we can have 'A' mean 'F' if 
F-contiguous, 'C' otherwise: [...]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Well, raw_memory() would avoid a copy, which is useful.

As for tobytes(), if we want to follow NumPy, we can have 'F' mean if 
F-contiguous, 'C' otherwise:

>>> a = np.arange(12, dtype='int8').reshape((3,4))  
>>>
>>> a.tobytes('A')  
>>>
b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b'
>>> a.tobytes('A') == a.T.tobytes('A')  
>>>
True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Stefan Krah]

Stefan Krah  added the comment:

raw_bytes() is also possible of course. I assume it would do nothing and just 
dump the memory.

Or tobytes('F') AND tobytes('raw').

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Stefan Krah]

Stefan Krah  added the comment:

Yes, it's modeled after NumPy's tobytes():

>>> x = np.array(list(range(6)), dtype="int8").reshape(2,3)
>>> x.tobytes()
b'\x00\x01\x02\x03\x04\x05'
>>> x.T.tobytes()
b'\x00\x03\x01\x04\x02\x05'
>>> 
>>> 
>>> memoryview(x).tobytes()
b'\x00\x01\x02\x03\x04\x05'
>>> memoryview(x.T).tobytes()
b'\x00\x03\x01\x04\x02\x05'


I guess the reason is that without a type it's easier to serialize the logical 
array by default, so you can always assume C when you read back.



NumPy also has an 'F' parameter though that flips the order:

>>> x.tobytes('F')
b'\x00\x03\x01\x04\x02\x05'

It would be possible to add this to memoryview as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35845] Can't read a F-contiguous memoryview in physical order

[Antoine Pitrou]

New submission from Antoine Pitrou :

This request is motivated in detail here:
https://github.com/python/peps/pull/883#issuecomment-458290745

In short: in C, when you have a Py_buffer, you can directly read the memory in 
whatever order you want (including physical order).  It is not possible in pure 
Python, though.  Somewhat unintuitively, memoryview.tobytes() as well as 
bytes(memoryview) read bytes in *logical* order, even though it flattens the 
dimensions and doesn't keep the original type.  Logical order is different from 
physical order for Fortran-contiguous arrays.

One possible way of alleviating this would be to offer a memoryview.transpose() 
method, similar to the Numpy transpose() method (see 
https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.transpose.html).

One could also imagine a memoryview.to_c_contiguous() method.

Or even: a memoryview.raw_memory() method, that would 1) flatten dimensions 2) 
cast to 'B' format 3) keep physical order.

--
components: Interpreter Core
messages: 334491
nosy: pitrou, skrah
priority: normal
severity: normal
stage: needs patch
status: open
title: Can't read a F-contiguous memoryview in physical order
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com