I have been looking at a problem that has arisen in recent versions,
particularly when using some D3D games, in which the virtual address space
above TASK_UNMAPPED_BASE becomes fragmented to the extent that eventually you
get an out of memory condition, even though you still have well over a
gigabyte free (even in a contiguous block!). This has led to some people
using the attached hack (first attachment) to get their D3D games to work.
After considering a lot of approaches to this problem that did not seem to get
anywhere productive, it seemed that the only way to resolve it would be to
get in between the application and any libraries, and the kernel. The
solution I came up with is somewhat rude, but if we assume that any libraries
that Wine uses only call mmap() and associated calls via the C library, then
it may well do the trick, plus make it possible to implement other virtual
memory APIs more effectively (and in a couple of cases eliminate calls to
wineserver).
If wine_preloader were extended to have its own implementation of all the
friends of mmap(), and to have its own implementation of the dynamic linker,
then in principle it could make sure only its mmap (and not the C library's)
is called. An even more aggressive approach might be to load the C library
and stick jumps into its mmap that redirect to the preloader's versions. The
preloader's mmap would keep track of mappings on its own and when it receives
an mmap with a start address of "NULL", decides on its own what base address
to use.
Any thoughts?
--- wine/libs/wine/mmap.c.orig 2006-01-27 04:58:28.000000000 -0900
+++ wine/libs/wine/mmap.c 2006-01-27 05:00:46.000000000 -0900
@@ -164,7 +164,26 @@
#endif /* (__svr4__ || __NetBSD__) && !MAP_TRYFIXED */
+static void *get_anon_mmap_null_address(size_t size)
+{
+ static int got_override = 0;
+ static void *low_alloc_ptr = NULL;
+ void * current_low_alloc_ptr;
+
+ if (!got_override)
+ {
+ low_alloc_ptr = (void*)0x10000000;
+ got_override = 1;
+ }
+ current_low_alloc_ptr = low_alloc_ptr;
+
+ if (low_alloc_ptr)
+ low_alloc_ptr += size;
+
+ return current_low_alloc_ptr;
+ }
+
/***********************************************************************
* wine_anon_mmap
*
@@ -212,6 +231,8 @@
return start;
#endif
}
+ if ((start == NULL) && !(flags & MAP_FIXED))
+ start = get_anon_mmap_null_address(size);
return mmap( start, size, prot, flags, fdzero, 0 );
#else
return (void *)-1;
7c000000-7c001000 r-xp 00001000 08:11 21
/home/troy/staticthing
7c001000-7c002000 rwxp 00001000 08:11 21
/home/troy/staticthing
ffffd000-ffffe000 rwxp ffffd000 00:00 0
ffffe000-fffff000 ---p 00000000 00:00 0
