[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Xiang Zhang]

Xiang Zhang added the comment:

Although Serhiy thinks we need a separate class for this but I still want to 
upload my patch first. Maybe some of it can be helpful later, or garbage.

I add a mmap_contains to fix the in operator's behaviour (I don't find the 
separate issue). I use the simplest search method which is O(m*n). Previously I 
thought it is not acceptable but I find out that mmap_gfind goes this way too.

By the way, only operations related to mmap_item are affected, which I can see 
is iteration and in (search does not need to iterate since there is find 
method), indexing is not affected. So maybe this does not break the backward 
compatibility that hard.

Hope no disturb.

--
keywords: +patch
Added file: http://bugs.python.org/file42647/mmap_bytearray_like.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Making iteration returns int is backward incompatible change. I afraid it is 
too later to do this. We lost a chance at a time of Python 3.0.

We need separate mmap class that behave more like 
bytes/bytearray/memoryview/sequence of 8-bit integers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Xiang Zhang]

Xiang Zhang added the comment:

Ho, I'm really curious to see the resolution. ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Berker Peksag]

Berker Peksag added the comment:

Thanks for taking a look at this, Xiang.

Like I said in msg263470, making the in operator work with mmap objects is out 
of scope for this issue and it should be handled in a separate issue (I already 
have a WIP patch, but please feel free to work on it).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Xiang Zhang]

Xiang Zhang added the comment:

I tried to write a patch to make mmap behave like bytearray more. Making 
iteration returns int is easy. But I am trapped in the contains operation. To 
support operation like b'aa' in b'aabbcc', we have to do a str in str search. I 
don't find any portable way except writing my own. bytes and bytearray use 
stringlib_find, but that is not reachable in a c module. Any advice?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Xiang Zhang]

Changes by Xiang Zhang :


--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Iterating a slice produces ints too (a slice is just a bytes object).

>>> import mmap, sys
>>> with open(sys.executable, 'rb') as f:
... mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
... print(next(iter(mm)))
... print(next(iter(mm[:10])))
... 
b'\x7f'
127

Seems this module taken little love when migrated to 3.0.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Berker Peksag]

Berker Peksag added the comment:

I don't think we can change this in 3.5 since it would break backward 
compatibility.

> Similarly the `in` operator seems to be broken; one could search for space 
> using `32 in bytesobj`, which would work for slices but not for the whole 
> mmap object.

Seems a reasonable request to me. Could you please open a separate issue?

--
components: +Extension Modules
nosy: +berker.peksag
priority: normal -> low
stage:  -> needs patch
type: behavior -> enhancement
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
nosy: +twouters
versions:  -Python 3.2, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26358] mmap.mmap.__iter__ is broken (yields bytes instead of ints)

[Antti Haapala]

New submission from Antti Haapala:

Just noticed when answering a question on StackOverflow 
(http://stackoverflow.com/q/35387843/918959) that on Python 3 iterating over a 
mmap object yields individual bytes as bytes objects, even though iterating 
over slices, indexing and so on gives ints

Example:

import mmap

with open('test.dat', 'rb') as f:
mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
for b in mm:
print(b)
# prints for example b'A' instead of 65
mm.close()

I believe this should be fixed for the sake of completeness - the documentation 
says that "Memory-mapped file objects behave like both bytearray and like file 
objects." - however the current behaviour is neither like a bytearray nor like 
a file object, and quite confusing.

Similarly the `in` operator seems to be broken; one could search for space 
using `32 in bytesobj`, which would work for slices but not for the whole mmap 
object.

--
messages: 260261
nosy: ztane
priority: normal
severity: normal
status: open
title: mmap.mmap.__iter__ is broken (yields bytes instead of ints)
type: behavior
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com