Hello!
I am a developer at Arm working on improving Cygwin compatibility on
the AArch64 architecture. I will soon be posting a series of bugfix
patches that allow additional unit tests to pass. Some (but not all) of
these patches are based on earlier work posted by Thirumalai Nagalingam,
itself authored by multiple contributors.
One of my patches is related to changing the memory layout that is
described in the file winsup/cygwin/local_includes/memory_layout.h,
and I am writing to ask for feedback on this change.
* Background/observed issue
---------------------------
Currently, the range from 0x800000000 to 0xa00000000 is reserved for
the cygheap. However, on Windows on Arm (AArch64), where mandatory
userspace ASLR is enforced, I have observed this region occasionally
being occupied by the PEB, TEB, and stack.
This can be reproduced with reasonably high probability using the
ltp/fork11.exe test. Below is an excerpt from a WinDbg !address
dump showing such a case:
BaseAddress EndAddress+1 RegionSize Type State
Protect Usage
--------------------------------------------------------------------------------------------------------------------------
+ 0`00000000 0`7ffe0000 0`7ffe0000 MEM_FREE
PAGE_NOACCESS Free
+ 0`7ffe0000 0`7ffe1000 0`00001000 MEM_PRIVATE MEM_COMMIT
PAGE_READONLY Other [User Shared Data]
+ 0`7ffe1000 1`a0000000 1`2001f000 MEM_FREE
PAGE_NOACCESS Free
+ 1`a0000000 1`a000f000 0`0000f000 MEM_MAPPED MEM_COMMIT
PAGE_READWRITE MappedFile "PageFile"
+ 1`a000f000 1`a0010000 0`00001000 MEM_FREE
PAGE_NOACCESS Free
+ 1`a0010000 1`a0019000 0`00009000 MEM_MAPPED MEM_COMMIT
PAGE_READWRITE MappedFile "PageFile"
+ 1`a0019000 1`a0020000 0`00007000 MEM_FREE
PAGE_NOACCESS Free
+ 1`a0020000 1`a0030000 0`00010000 MEM_MAPPED MEM_COMMIT
PAGE_READWRITE MappedFile "PageFile"
+ 1`a0030000 1`a0031000 0`00001000 MEM_MAPPED MEM_COMMIT
PAGE_READWRITE MappedFile "PageFile"
+ 1`a0031000 7`ffe00000 6`5fdcf000 MEM_FREE
PAGE_NOACCESS Free
+ 7`ffe00000 7`ffff5000 0`001f5000 MEM_PRIVATE MEM_RESERVE
Stack [~0; 2658.41b8]
7`ffff5000 7`ffff8000 0`00003000 MEM_PRIVATE MEM_COMMIT
PAGE_READWRITE | PAGE_GUARD Stack [~0; 2658.41b8]
7`ffff8000 8`00000000 0`00008000 MEM_PRIVATE MEM_COMMIT
PAGE_READWRITE Stack [~0; 2658.41b8]
+ 8`00000000 8`00300000 0`00300000 MEM_PRIVATE MEM_COMMIT
PAGE_READWRITE <unknown> [................]
8`00300000 a`00000000 1`ffd00000 MEM_PRIVATE MEM_RESERVE
<unknown>
+ a`00000000 a`00030000 0`00030000 MEM_PRIVATE MEM_COMMIT
PAGE_READWRITE <unknown> [................]
a`00030000 a`20000000 0`1ffd0000 MEM_PRIVATE MEM_RESERVE
<unknown>
+ a`20000000 78`3ee00000 6e`1ee00000 MEM_FREE
PAGE_NOACCESS Free
[...]
+ 186`d9c60000 7ff4`ee570000 7e6e`14910000 MEM_FREE
PAGE_NOACCESS Free
* Proposed change
-----------------
My proposal is to shift cygheap, user heap and mmap_storage to
the end of the largest FREE block of memory.
Based on empirical testing, this appears to work consistently and
reliably, at least on Windows 11 23H2, on both AArch64 (with
mandatory ASLR) and x86_64. Here is an example of this block on
x86_64 with ASLR disabled:
+ 1`0041e000 7ff4`fdec0000 7ff3`fdaa2000 MEM_FREE
PAGE_NOACCESS Free
Please see attachments for my proposed changes.
* Questions
-----------
I have not been able to find any authoritative documentation that
would describe the exact mapping of these ranges whether on x86_64
or AArch64, on Microsoft's website or elsewhere.
I would therefore appreciate any guidance from the community on:
* whether this relocation has potential downsides I may be missing,
* whether there are known constraints on placing cygheap/user heap
in this region,
* or whether there is a more appropriate approach to handling this
under mandatory ASLR.
With regard to shared-memory interactions, my understanding is that
this change should not introduce new issues. The Cygwin FAQ already
notes that mixing executables from different Cygwin installations is
not a supported configuration
(https://cygwin.com/faq.html#faq.using.multiple-copies), and the
proposed change does not alter that assumption.
I look forward to any feedback or suggestions.
Best regards,
Igor Podgainoi
Arm
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
diff --git a/winsup/cygwin/local_includes/memory_layout.h b/winsup/cygwin/local_includes/memory_layout.h
index f8ec80bb5..9cbcc4339 100644
--- a/winsup/cygwin/local_includes/memory_layout.h
+++ b/winsup/cygwin/local_includes/memory_layout.h
@@ -36,20 +36,20 @@ details. */
/* That's where the cygheap is located. CYGHEAP_STORAGE_INITIAL defines the
end of the initially committed heap area. */
-#define CYGHEAP_STORAGE_LOW 0x800000000UL
-#define CYGHEAP_STORAGE_INITIAL 0x800300000UL
-#define CYGHEAP_STORAGE_HIGH 0xa00000000UL
+#define CYGHEAP_STORAGE_LOW 0x0ff700000000UL
+#define CYGHEAP_STORAGE_INITIAL 0x0ff700300000UL
+#define CYGHEAP_STORAGE_HIGH 0x0ff900000000UL
/* This is where the user heap starts. There's no defined end address.
The user heap pontentially grows into the mmap arena. However,
the user heap grows upwards and the mmap arena grows downwards,
so there's not much chance to meet unluckily. */
-#define USERHEAP_START 0xa00000000UL
+#define USERHEAP_START 0x0ff900000000UL
/* The memory region used for memory maps. Mmaps grow downwards.
Set the lowest address to leave ~32 Gigs for heap. */
-#define MMAP_STORAGE_LOW 0x001000000000UL
-#define MMAP_STORAGE_HIGH 0x700000000000UL
+#define MMAP_STORAGE_LOW 0x0fff00000000UL
+#define MMAP_STORAGE_HIGH 0x7fef00000000UL
/* Default number of pages used as thread stack guard pages. */
#define DEFAULT_GUARD_PAGE_COUNT 3
