Full_Name: David Teller Version: 1.7.1 OS: Windows XP Submission from: (NULL) (12.110.141.194)
I've noticed several issues with the R memory management under Windows. It frequently runs out of memory, even when there is plenty of available memory on the system and even plenty of free process virtual address space. This tends to happen far more often when allocation a number of large objects early in the processing. Investigating the malloc.c routines indicates that when expanding the heap to allocate a large object that will not fit, the windows sbrk emulation routine calls an internal routine findregion with the object size (rounded up for overhead). This routine essentially searches the virtual address space for an unused place large enough to put the object, looking towards higher addresses, until it finds a suitable location and always starting at the end of the last chunk allocated. The issue is that any available space that is not large enough for the object in question is bypassed and can NEVER be used by the memory allocator. This can lead to a huge amount of address space wastage if several large objects are allocated early in the process. It seems that the search cannot just be restarted at the beginning of memory each time because the malloc implementation assumes that each chunk of memory allocated from the system will be at a higher address than the last. I have several possible solutions to this and am thinking about implementing one of them: 1) Completely scrap the routines in malloc.c and replace them with a thin wrapper around the core Windows memory allocation routines. 2) Attempt to implement a windows version of mmap() and enable that feature for large allocations. 3) Attempt to clean up the malloc routines so they are less picky about the address ordering of chunks allocated from the system. Any feedback would be appreciated. I don't see any other bugs on this issue and don't want to duplicate efforts if someone else is already working on this. - Dave Teller ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-devel
