[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Laurent Gautier]

Laurent Gautier  added the comment:

Bump.

I am still stumbling on that issue. I have a workaround by importing numpy but 
that's a rather large dependency for that one little thing.

I see that the related issue https://bugs.python.org/issue35845 opened later 
has later discussion entries.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Laurent Gautier]

Laurent Gautier  added the comment:

Bump.

Is there anything I could do to move this forward (as in write and submit a 
patch for review) ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Stefan Krah]

Change by Stefan Krah :


--
assignee:  -> skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Laurent Gautier]

Laurent Gautier  added the comment:

> General support for strides in cast(), i.e. a zero-copy view for
> non-contiguous arrays does not seem possible because buf.ptr is
> moved around. Even NumPy does not support that:

I'd be happy enough with zero-copy `cast()` of f-continguous arrays
along with the parameter `shape`, as this makes interfacing with
orginally-in-FORTRAN C libraries through ctypes or cffi possible
(without having to write a C-extension).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Stefan Krah]

Stefan Krah  added the comment:

It seems reasonable to support f-contiguous for cast() and tobytes().
For tobytes() it's implemented in the issue that Antoine linked to.


General support for strides in cast(), i.e. a zero-copy view for
non-contiguous arrays does not seem possible because buf.ptr is
moved around. Even NumPy does not support that:

>>> x = np.array([1,2,3])
>>> x.view('B')
array([1, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0,
   0, 0], dtype=uint8)
>>> 
>>> x[::-1].view('B')
Traceback (most recent call last):
  File "", line 1, in 
ValueError: To change to a dtype of a different size, the array must be 
C-contiguous
>>> 
>>> y = x.astype('B')
>>> y.flags['OWNDATA'] # It's a copy.
True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Nick Coghlan]

Nick Coghlan  added the comment:

+1 for Antoine's comment - while our approach with memoryview has generally 
been "If you're doing serious work with n-dimensional data, you still need 
NumPy (or an equivalent)", we've also been open to borrowing more NumPy 
behaviours for memoryview as particular pain points arise, and I think that 
applies here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I think feature creep is ok if it stems from user needs.

Slighty related, but simpler, is https://bugs.python.org/issue35845

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Stefan Krah]

Stefan Krah  added the comment:

CC Antoine and Nick.

I think we can do it, but we'd need cast(shape=[2,3], order='F')
to allow casting back.

The only practical objections are feature creep. To preserve
symmetry with tobytes(), we'd need to add tobytes('F') (and
tobytes('A')).

--
nosy: +ncoghlan, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[mattip]

mattip  added the comment:

> the original decision to exclude non 'C' views was deliberate

Seems this is reflected in the code:

```
a = np.array([[0, 1, 2], [3, 4, 5]])
mv = memoryview(a.T)
mv.f_contiguous 
# True
mv.cast('i', (3, 2))
# TypeError: memoryview: casts are restricted to C-contiguous views
```

Is there any interest in revisiting that discussion? It seems the buffer 
protocol could allow more flexibility wrt strides and contiguous flags. Do you 
have a link to the discussion where this was rejected?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Laurent Gautier]

Laurent Gautier  added the comment:

Wait. Isn't a `memoryview` memerely a Python object for a buffer inferface, 
whatever its valid attributes or flags might be ?

The perceived oddness that lead to the addition of the keyword 'shape' was a 
good initial instinct that something was off, but this is an incomplete 
workaround .

If the rationale was to follow what `tobytes` is doing, this delegates the 
justification for excluding non 'C' views it. Then I do not understand the 
rationale behind `memoryview.tobytes`'s exclusive relationshop to C-contiguous 
arrays. A memmoryview is a window on a memory region (a Python buffer), and one 
would expect `tobytes` to just return bytes for it (in whatever bytes/strides) 
the memoryview is originally in.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Stefan Krah]

Stefan Krah  added the comment:

memoryview.cast() was originally meant to be a faster version of tobytes(), 
which always converts to C-contiguous.

The 'shape' keyword was added because it is odd if you can cast from ND-C to 
1D-Bytes but not back.

I'm not sure if we should introduce that feature, just pointing out that the 
original decision to exclude non 'C' views was deliberate.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Laurent Gautier]

Laurent Gautier  added the comment:

Bump.

What are the next steps here ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Laurent Gautier]

Laurent Gautier  added the comment:

Wouldn't a contiguity argument ('C' or 'F') be simpler ?
 
(Independently, an argument strides is likely also missing from "cast").

Do you know what are the next possible steps here ? Bring this to the 
python-dev list ? Submit a patch ?

--
components: +Extension Modules, Library (Lib) -Interpreter Core

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[mattip]

mattip  added the comment:

Sorry, I meant a "strides" keyword. "shape" is already a valid keyword

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Laurent Gautier]

Laurent Gautier  added the comment:

@mattip : do you mean that it can currently be achieved by calling `cast` with 
a specific shape parameter ? If the case, how so ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[mattip]

mattip  added the comment:

This could be done via a `shape` kwarg to `cast`

--
nosy: +mattip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Stefan Krah]

Change by Stefan Krah :


--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34778] Memoryview for column-major (f_contiguous) arrays from bytes impossible to achieve

[Laurent Gautier]

New submission from Laurent Gautier :

The buffer protocol is accounting for the row-major or column-major arrays, and 
that information is shown in the attributes, `c_contiguous` and `f_contiguous` 
respectively, of a memoryview object.

Using the method `cast` allows one to specify a shape but does not allow
to specify whether row or column major:

```
# column-major 3x2 array of bytes that was serialized
b = bytearray([1,2,3,4,5,6])

mv = memoryview(b)
mv_b = mv.cast('b', shape=(3,2))
```

The result object is believed to be row-major and little can be done
to correct it:


```
>>> mv_int.c_contiguous
True
>>> mv_int.c_contiguous = False
AttributeError: attribute 'c_contiguous' of 'memoryview' objects is not writable
```

--
components: Interpreter Core
messages: 326167
nosy: lgautier
priority: normal
severity: normal
status: open
title: Memoryview for column-major (f_contiguous) arrays from bytes impossible 
to achieve
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com