Thanks, Erik.
StefanK
On 2019-11-22 15:24, Erik Joelsson wrote:
Build change looks good.
/Erik
On 2019-11-21 13:11, Stefan Karlsson wrote:
Hi,
I'm looking for a review for this tiny build change:
https://cr.openjdk.java.net/~stefank/8233299/webrev.01/make/autoconf/hotspot.m4.udiff.html
Thanks,
StefanK
-------- Forwarded Message --------
Subject: RFR: 8233299: Implementation: JEP 365: ZGC on Windows
Date: Thu, 31 Oct 2019 11:18:20 +0100
From: Stefan Karlsson <stefan.karls...@oracle.com>
To: hotspot-gc-dev <hotspot-gc-...@openjdk.java.net>
Hi all,
Please review this patch to add ZGC support on Windows.
https://cr.openjdk.java.net/~stefank/8233299/webrev.01/
https://bugs.openjdk.java.net/browse/JDK-8233299
As mentioned in the JEP (https://openjdk.java.net/jeps/365), there
were some preparation patches that needed to go in to pave the way for
this patch:
8232601: ZGC: Parameterize the ZGranuleMap table size
8232602: ZGC: Make ZGranuleMap ZAddress agnostic
8232604: ZGC: Make ZVerifyViews mapping and unmapping precise
8232648: ZGC: Move ATTRIBUTE_ALIGNED to the front of declarations
8232649: ZGC: Add callbacks to ZMemoryManager
8232650: ZGC: Add initialization hooks for OS specific code
8232651: Add implementation of os::processor_id() for Windows
... they have all been pushed now.
One important key-point to this implementation is to use the new
Windows APIs that support reservation and mapping of memory through
"placeholders": VirtualAlloc2, VirtualFreeEx, MapViewOfFile3, and
UnmapViewOfFile2. These functions are available starting from version
1803 of Windows 10 and Windows Server. ZGC will lookup these symbols
to determine if the Windows version supports these functions.
Correlating the text in the JEP with the code:
* '"Support for multi-mapping memory". ZGC's use of colored pointers
requires support for heap multi-mapping, so that the same physical
memory can be accessed from multiple different locations in the
process address space. On Windows, paging-file backed memory provides
physical memory with an identity (a handle), which is unrelated to the
virtual address where it is mapped. Using this identity allows ZGC to
map the same physical memory into multiple locations.'
We commit memory via paging file mappings and map views into that memory.
The function ZMapper::create_and_commit_paging_file_mapping uses
CreateFileMappingW with SEC_RESERVE to create this mapping,
MapViewOfFile3 to map a temporary view into the mapping, VirtualAlloc2
to commit the memory, and then UnmapViewOfFile2 to unmap the view.
The reason to use SEC_RESERVE and the extra VirtualAlloc2, instead of
SEC_COMMIT, is to ensure that the later multi-mappings of committed
file mappings don't fail under low-memory situations. Earlier
prototypes used SEC_COMMIT and saw these kind of OOME errors when
mapping new views to already committed memory. The current
platform-independent ZGC code isn't prepared to handle OOME errors
when mapping views, so we chose this solution.
MapViewOfFile3 is then used to multi-map into the committed memory.
* '"Support for mapping paging-file backed memory into a reserved
address space". The Windows memory management API is not as flexible
as POSIX's mmap/munmap, especially when it comes to mapping file
backed memory into a previously reserved address space region. To do
this, ZGC will use the Windows concept of address space placeholders.
The placeholder concept was introduced in version 1803 of Windows 10
and Windows Server. ZGC support for older versions of Windows will not
be implemented.'
Before the placeholder APIs there was no way to first reserve a
specific virtual memory range, and then map a view of a committed
paging file over that range. The VirtuaAlloc function could be used to
first reserve and then commit anonymous memory, but nothing similar
existed for mapped views. Now with placeholders, we can create a
placeholder reservation of memory with VirtualAlloc2, and then replace
that reservation with MapViewOfFile3. When memory is unmapped, we can
use UnmapViewOfFile2 to "preserve" the placeholder memory reservation.
* '"Support for mapping and unmapping arbitrary parts of the heap".
ZGC's heap layout in combination with its dynamic sizing (and
re-sizing) of heap pages requires support for mapping and unmapping
arbitrary heap granules. This requirement in combination with Windows
address space placeholders requires special attention, since
placeholders must be explicitly split/coalesced by the program, as
opposed to being automatically split/coalesced by the operating system
(as on Linux).'
Half of the preparation patches were put in place to support this.
When replacing a placeholder with a view of the backing file, we need
to exactly match the address and size of a placeholder. Also, when
unmapping a view, we need to exactly match the address and size of the
view, and replace it with a placeholder.
To make it easier to map and unmap arbitrary parts of the heap, we
split reserved memory into ZGranuleSize-sized placeholders. So,
whenever we perform any of these operations, we know that any given
memory range could be dealt with as a number of granules.
When memory is reserved, but not mapped, it is registered in the
ZVirtualMemoryManager. It splits memory into granule-sized placholders
when reserved memory is fetched, and coalesces placeholders when
reserved memory is handed back.
* '"Support for committing and uncommitting arbitrary parts of the
heap". ZGC can commit and uncommit physical memory dynamically while
the Java program is running. To support these operations the physical
memory will be divided into, and backed by, multiple paging-file
segments. Each paging-file segment corresponds to a ZGC heap granule,
and can be committed and uncommitted independently of other segments.'
Just like we can map and unmap in granules, we want to be able to
commit and uncommit memory in granules. You can see how memory is
committed and uncommitted in granules in
ZBackingFile::commit_from_paging_file and
ZBackingFile::uncommit_from_paging_file. Each committed granule is
associated with one registered handle. When memory for a granule is
uncommitted, the handle is closed. At this point, no views exist to
the mapping and the memory is handed back to the OS.
Final point about ZPhysicalMemoryBacking. We've tried to make this
file similar on all OSes, with the hope to be able to combine them
when both the Windows and macOS ports have been merged.
Thanks,
StefanK
