[issue12562] calling mmap twice fails on Windows

[Tim Golden]

Tim Golden  added the comment:

(switching stage to resolved because it's closed/rejected; sorry for the noise)

--
stage:  -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12562] calling mmap twice fails on Windows

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Vlad, thank you for the diagnosis.
Indeed by passing a different tagname (or no tagname at all), the problem 
doesn't occur.
Since mmap.mmap() matches the semantics chosen by Microsoft here, I tend to 
think this is not a bug; besides, fixing it would probably be intricate.

--
resolution:  - rejected
status: open - pending
versions: +Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12562
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12562] calling mmap twice fails on Windows

[Piotr Zolnierczuk]

Piotr Zolnierczuk piotr.zolnierc...@gmail.com added the comment:

OK. I will work around it.
 
I was using 'mapping object of a specified size that is backed by the system 
paging file instead of by a file in the file system' - map-handle == 
INVALID_HANDLE_VALUE (-1), i.e. shared memory for inter-process communication 
(not my choice)

A use case was when all process were stopped and one wanted to start fresh 
using the same tag but with a bigger size. 

BTW, here's the result of the following script in Python 2.5
 import mmap
 map1 = mmap.mmap(fileno=-1, tagname='MyData', length=4096)
 map1[0] = 'a'
 map1[4095]='b'
 print len(map1), map1[0], map1[4095]
4096 a b
 map2 = mmap.mmap(fileno=-1, tagname='MyData', length=8192)
 print len(map2), map2[0], map2[4095]
8192 a b

which would indicate that it does 'resize' and preserves the data.



which means t

--
status: pending - open

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12562
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12562] calling mmap twice fails on Windows

[Piotr Zolnierczuk]

Piotr Zolnierczuk piotr.zolnierc...@gmail.com added the comment:

Just looked into my partner C++ code and he's using it very much like in 
Python 2.5:

m_obj-map_handle = CreateFileMapping (INVALID_HANDLE_VALUE,
NULL,   PAGE_READWRITE, 
 0,
pageSize,   
 tagName);

m_obj-data = (char *) MapViewOfFile (m_obj-map_handle,
  FILE_MAP_ALL_ACCESS,  
  0,
  0,
  0);

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12562
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12562] calling mmap twice fails on Windows

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Note that multiprocessing can abstract this kind of things for you:
http://docs.python.org/library/multiprocessing#sharing-state-between-processes
http://docs.python.org/library/multiprocessing#module-multiprocessing.sharedctypes

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12562
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12562] calling mmap twice fails on Windows

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12562
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12562] calling mmap twice fails on Windows

[Vlad Riscutia]

Vlad Riscutia riscutiav...@gmail.com added the comment:

Reason you are seeing the failure for this is following change from 2.5 in 
mmapmodule.c (:1109):

m_obj-data = (char *) MapViewOfFile(m_obj-map_handle,
 dwDesiredAccess,
 0,
 0,
 0);

changed to mmapmodule.c (:1414 in 3.3):

m_obj-data = (char *) MapViewOfFile(m_obj-map_handle,
 dwDesiredAccess,
 off_hi,
 off_lo,
 m_obj-size);

Previously size wasn't passed to MapViewOfFile. Passing new size to 
MapViewOfFile greater than the size of previous map causes an error. 

This seems to be by design. From MSDN:

MapViewOfFile:

dwNumberOfBytesToMap [in]
The number of bytes of a file mapping to map to the view. All bytes must be 
within the maximum size specified by CreateFileMapping. If this parameter is 0 
(zero), the mapping extends from the specified offset to the end of the file 
mapping.

CreateFileMapping:

lpName [in, optional]
The name of the file mapping object.

If this parameter matches the name of an existing mapping object, the function 
requests access to the object with the protection that flProtect specifies.

So on second call, CreateFileMapping will get back the previous mapping object, 
which has 4096 bytes of memory mapped. MapViewOfFile will try to map beyond its 
limit and get an error.

I am curious how resizing worked before. I tried passing size=0 to 
MapViewOfFile on second call (length=8192) then call VirtualQuery on the 
returned map, which can query the size of the buffer. Size is still 4096. So 
even if length=8192 and we call CreateFileMapping with this length, it will 
return the previous 4096 byte-buffer.

This looks to me like an issue which existed until Python 2.5, namely this 
error was silenced and returned map was still 4096 bytes, just claiming to be 
8192. The way it is behaving now seems to be the correct way.

--
nosy: +vladris

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12562
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12562] calling mmap twice fails on Windows

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +brian.curtin, pitrou, tim.golden

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12562
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12562] calling mmap twice fails on Windows

[Piotr Zolnierczuk]

New submission from Piotr Zolnierczuk piotr.zolnierc...@gmail.com:

Hi,
I am trying to migrate from Python 2.5 to Python 2.7 I found though the mmap 
behaves differently on Windows XP between the two versions. It boils down to 
the following code:
import mmap
map1 = mmap.mmap(fileno=0, tagname='MyData', length=4096)
map2 = mmap.mmap(fileno=0, tagname='MyData', length=8192)

It runs fine (so I can resize shared memory) on XP with 2.5.4, but when 
running on 2.7.2 I get the following error

Traceback (most recent call last):
  File D:\Workspace\memmap_test.py, line 3, in module
map2 = mmap.mmap(fileno=0, tagname='MyData', length=8192)
WindowsError: [Error 5] Access is denied

--
messages: 140349
nosy: zolnie
priority: normal
severity: normal
status: open
title: calling mmap twice fails on Windows
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12562
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12562] calling mmap twice fails on Windows

[Piotr Zolnierczuk]

Changes by Piotr Zolnierczuk piotr.zolnierc...@gmail.com:


--
components: +Windows
type:  - behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12562
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com