George Sakkis wrote:
> It's around 400MB.
On Windows you may not be able to map a file of this size into memory
because of virtual address space fragmentation. A Win32 process has
only 2G of virtual address space, and DLLs tend to get scattered
through out that address space.
> As I said, I cann
Nick Craig-Wood schrieb:
> So presumably it is python generating a MemoryError. It is asking for
> a new bit of memory and it is failing so it throws a MemoryError.
>
> Could memory allocation under windows be affected by a large chunk of
> mmap()ed file which is physically swapped in at the time
Laszlo Nagy schrieb:
>
>> In fact, memory that is read in because of mmap should *never* cause
>> a MemoryError.
> This is certainly not true. You can run out of virtual address space by
> reading data from a memory mapped file.
That is true, but not what I said. I said you cannot run out of mem
> It's around 400MB. As I said, I cannot reproduce the MemoryError
> locally since I have 1GB physical space but IIRC the user who reported
> it had less. Actually I am less concerned about whether a MemoryError
> is raised or not in this case and more about the fact that even if
> there's no exce
> In fact, memory that is read in because of mmap should *never* cause
> a MemoryError.
This is certainly not true. You can run out of virtual address space by
reading data from a memory mapped file.
> Python calls MapViewOfFile when mmap.mmap is invoked,
> at which point the operating commits t
Dennis Lee Bieber wrote:
> On 21 Jan 2007 13:32:19 -0800, "George Sakkis" <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
>
> >
> > The file is written once and then opened as read-only, there's no
> > flushing. So if caching is completely up to the OS, I take it that my
>
>
George Sakkis <[EMAIL PROTECTED]> wrote:
> The file is written once and then opened as read-only, there's no
> flushing. So if caching is completely up to the OS, I take it that my
> options are either (1) modify my algorithms so that they work in
> fixed-size batches instead of arbitrarily lon
Martin v. Löwis <[EMAIL PROTECTED]> wrote:
> In fact, memory that is read in because of mmap should *never* cause
> a MemoryError. Python calls MapViewOfFile when mmap.mmap is invoked,
> at which point the operating commits to providing that much address
> space to the application, along with b
George Sakkis schrieb:
>> You must be misinterpreting what you are seeing. It's the operating
>> system that decides what part of a memory-mapped file are held in
>> memory, and that is certainly not without limits.
>
> Sure; what I meant was that that whatever the limit is, it's high
> enough tha
Martin v. Löwis wrote:
> George Sakkis schrieb:
> > I've been trying to track down a memory leak (which I initially
> > attributed erroneously to numpy) and it turns out to be caused by a
> > memory mapped file. It seems that mmap caches without limit the chunks
> > it reads, as the memory usage g
George Sakkis schrieb:
> I've been trying to track down a memory leak (which I initially
> attributed erroneously to numpy) and it turns out to be caused by a
> memory mapped file. It seems that mmap caches without limit the chunks
> it reads, as the memory usage grows to several hundreds MBs accor
Nick Craig-Wood wrote:
> George Sakkis <[EMAIL PROTECTED]> wrote:
> > I've been trying to track down a memory leak (which I initially
> > attributed erroneously to numpy) and it turns out to be caused by a
> > memory mapped file. It seems that mmap caches without limit the chunks
> > it reads,
George Sakkis <[EMAIL PROTECTED]> wrote:
> I've been trying to track down a memory leak (which I initially
> attributed erroneously to numpy) and it turns out to be caused by a
> memory mapped file. It seems that mmap caches without limit the chunks
> it reads, as the memory usage grows to seve
I've been trying to track down a memory leak (which I initially
attributed erroneously to numpy) and it turns out to be caused by a
memory mapped file. It seems that mmap caches without limit the chunks
it reads, as the memory usage grows to several hundreds MBs according
to the Windows task manage
14 matches
Mail list logo
