[issue35686] BufferError with memory.release()

[Stefan Krah]

Stefan Krah  added the comment:

Eryk Sun:

Yes, the behavior is technically not guaranteed. I'm not sure about
memoryview(x, start, stop, step) but I'll keep it in mind.


Thomas Waldmann:
> do you think this is as good as it gets for this kind of code?

I guess so, there's a lot going on in that code fragment.

--
components: +Interpreter Core
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Thomas Waldmann]

Thomas Waldmann  added the comment:

https://github.com/borgbackup/borg/pull/4247/files

this is the current code i have for this (it is not as simple there as in the 
minimal code samples i attached here).

i meanwhile think i can not use a contextmanager there.

do you think this is as good as it gets for this kind of code?

if you all think this is expected behaviour for the python contextmanagers and 
can't be done better, this issue can be closed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Eryk Sun]

Eryk Sun  added the comment:

> Or, obviously:
> 
> with open(fn, 'rb') as fd:
> with mmap.mmap(fd.fileno(), 0, access=mmap.ACCESS_READ) as mm:
> with memoryview(mm)[:2] as data:
> print(data)

Doesn't this rely on the immediate finalization of the unreferenced memoryview 
instance that creates the slice? It would be nice if memoryview supported 
alternate constructors memoryview(obj, stop) and memoryview(obj, start, stop[, 
step]).

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Stefan Krah]

Stefan Krah  added the comment:

Or, obviously:

with open(fn, 'rb') as fd:
with mmap.mmap(fd.fileno(), 0, access=mmap.ACCESS_READ) as mm:
with memoryview(mm)[:2] as data:
print(data)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Stefan Krah]

Stefan Krah  added the comment:

s/on export/one export/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Stefan Krah]

Stefan Krah  added the comment:

Well, the problem in b) is that data[:2] creates a new memoryview,
so the underlying ManagedBufferObject now has two exports:

  - One from the context manager.

  - The second from the slice.

So memoryview.__exit__() decrements on export, but the second one
is hanging.

This actually works as expected because the ManagedBufferObject
cannot know that it could also release the slice. That's what I
meant by saying that it's the application's responsibility to
release all views that are based on the context manager's view.


One way of doing so would be this:

with open(fn, 'rb') as fd:
with mmap.mmap(fd.fileno(), 0, access=mmap.ACCESS_READ) as mm:
with memoryview(mm) as data:
with data[:2] as theslice:
print(theslice)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Thomas Waldmann]

Thomas Waldmann  added the comment:

I see 2 issues here:

1. I would expect that the CM approach (b) behaves equivalent to try/finally 
(c), but it does not and even causes an exception.

2. The traceback given in the BufferError (in (b)) is misleading, it points to 
the "print", but should rather point to mmap.__exit__).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Thomas Waldmann]

Thomas Waldmann  added the comment:

hmm, issue tracker does not make it easy to see which file was uploaded for 
which comment, so:

a: original code, works, potentially problematic exception handling (memoryview 
not always being released)

b: using the memoryview context manager, fails with BufferError

c: using try/finally to make sure memoryview is released: works

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Thomas Waldmann]

Thomas Waldmann  added the comment:

with try/finally: works.

--
Added file: https://bugs.python.org/file48033/issue35686c.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Thomas Waldmann]

Thomas Waldmann  added the comment:

with context manager, does not work.

--
Added file: https://bugs.python.org/file48032/issue35686b.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Stefan Krah]

Stefan Krah  added the comment:

The behavior seems to be correct to me: If there are exports, the memoryview 
cannot be released.

The application needs to ensure that release() is not called when
there are exports left.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Thomas Waldmann]

Thomas Waldmann  added the comment:

working, but potentially problematic because .release is not always called 
(e.g. in case of an exception).

--
Added file: https://bugs.python.org/file48031/issue35686a.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35686] BufferError with memory.release()

[Stefan Krah]

Stefan Krah  added the comment:

We use "crash" for segmentation fault, but this appears to be a regular 
traceback that includes BufferError.

The BufferError message appears to be from mmapmodule.c.

--
title: memoryview contextmanager causing strange crash -> BufferError with 
memory.release()
type: crash -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com