Aleksey, this is COOL! Will follow your instruction to build it.
Thanks, xiaofeng
On 4/27/07, Aleksey Shipilev <[EMAIL PROTECTED]> wrote:
Hi, Xiao-Feng!
On 4/27/07, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
> This means GCv5 was not thoroughly tested with 1.5GB heap size yet. I
> will learn from Sergey on how to build DRLVM for 1.5GB heap size and
> go more testing then after.
I will explain the trick "How to get DRL VM acquire >1Gb of heap".
The problem is predefined base address in some of the libraries: they
are going to load at predefined location in system memory thus causing
fragmentation of possible heap space. Since there are problems on
allocating non-continuous heap, there's no possibility to allocate big
chunk of memory. So we will need to relocate some libraries to another
location.
This time we have only by-hand solution, which could be
machine-dependent. The idea is simple: try to allocate as much as we
can and see what blocks us. I've used the simple test:
public class HeapTest {
public static void main(String args[]) throws Exception {
System.out.println("HeapTest started");
System.in.read();
}
}
Then I run this test with max of possible heap:
harmony-hdk-r532358/jdk/jre/bin/java -Xms900M -Xmx900M
-XX:vm.dlls=gc_gen.dll -XX:gc.use_large_page=true HeapTest
...and see the DLL distribution across the memory: you could use
ProcessExplorer from SysInternals.com to obtain that list. I have this
picture:
Name Base Size
unicode.nls 0x260000 0x16000
locale.nls 0x280000 0x34000
sortkey.nls 0x2C0000 0x41000
sorttbls.nls 0x310000 0x6000
ctype.nls 0x330000 0x3000
zlib1.dll 0x3A0000 0x13000
odbc32.dll 0x3C0000 0x3D000
java.exe 0x400000 0xD000
harmonyvm.dll 0x510000 0x424000
dbghelp.dll 0x940000 0xA8000
odbcint.dll 0x11E0000 0x17000
em.dll 0x1330000 0x40000
jitrino.dll 0x1380000 0x410000
gc_gen.dll 0x17A0000 0x2C000
hysig.dll 0x17E0000 0x6000
hytext.dll 0x17F0000 0x6000
hyzlib.dll 0x1E70000 0x13000
vmi.dll 0x1E90000 0x6000
hynio.dll 0x1EA0000 0x6000
hyluni.dll 0x1EB0000 0x23000
hyarchive.dll 0x1EE0000 0xD000
icuinterface34.dll 0x25F0000 0x17000
hythr.dll 0x10000000 0x407000
hyprt.dll 0x11100000 0x18000
[ ------------------ here goes the chunk ------------------ ]
icuuc34.dll 0x4A800000 0xC8000
icuin34.dll 0x4A900000 0xAA000
icudt34.dll 0x4AD00000 0x870000
[ --------------- and here goes the chunk -------------- ]
mswsock.dll 0x71B20000 0x41000
ws2help.dll 0x71BF0000 0x8000
ws2_32.dll 0x71C00000 0x17000
comdlg32.dll 0x762B0000 0x4A000
userenv.dll 0x76920000 0xC4000
psapi.dll 0x76B70000 0xB000
secur32.dll 0x76F50000 0x13000
user32.dll 0x77380000 0x92000
comctl32.dll 0x77420000 0x103000
comctl32.dll 0x77530000 0x97000
version.dll 0x77B90000 0x8000
msvcrt.dll 0x77BA0000 0x5A000
gdi32.dll 0x77C00000 0x48000
rpcrt4.dll 0x77C50000 0x9F000
shlwapi.dll 0x77DA0000 0x52000
kernel32.dll 0x77E40000 0x102000
advapi32.dll 0x77F50000 0x9C000
msvcr71.dll 0x7C340000 0x56000
ntdll.dll 0x7C800000 0xC0000
shell32.dll 0x7C8D0000 0x803000
Let's try to merge these chunks together. We will use the editbin
utility from MS Platform SDK. I checked that editbin is on my $PATH
and then run the following script:
editbin /LARGEADDRESSAWARE java.exe
editbin /LARGEADDRESSAWARE /rebase:base=0x84000000 hythr.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x84500000 hysig.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x84550000 hyprt.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x84600000 hyzlib.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x84650000 hytext.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x84700000 vmi.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x84750000 hyluni.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x84800000 hyarchive.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x84850000 hynio.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x84900000 hycharset.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x85500000 gc_cc.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x85500000 gc_gen.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x85600000 harmonyvm.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x86100000 zlib1.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x86200000 em.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x86300000 jitrino.dll
editbin /LARGEADDRESSAWARE /rebase:base=0x87000000 hysecurity.dll
editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87030000 icuuc34.dll
editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87100000 icudt34.dll
editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87200000 icuin34.dll
editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87300000 icuin34.dll
editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87400000 icuin34.dll
editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87500000 ICUInterface34.dll
either on jre/bin and jre/bin/default directories.
The idea is simple too: we are moving the libraries at the end of
memory and base them there. Note that this script is really OS/build
dependent since the initial distribution is unknown.
After applying these transformations I was able to run the test again
with larger heap:
$ harmony-hdk-r532358/jdk/jre/bin/java -Xms1700M -Xmx1700M
-XX:vm.dlls=gc_gen.dll -XX:gc.use_large_page=true HeapTest
GC use large pages.
HeapTest started
Horray! Then I make sure that it worked out fine:
Name Base Size
unicode.nls 0x260000 0x16000
locale.nls 0x280000 0x34000
sortkey.nls 0x2C0000 0x41000
sorttbls.nls 0x310000 0x6000
ctype.nls 0x330000 0x3000
java.exe 0x400000 0xD000
odbcint.dll 0xCB0000 0x17000
[ ------------------ a BI-I-I-I-G chunk here --------------- ]
icuinterface34.dll 0x71AA0000 0x17000
mswsock.dll 0x71B20000 0x41000
ws2help.dll 0x71BF0000 0x8000
ws2_32.dll 0x71C00000 0x17000
icuin34.dll 0x72520000 0xAA000
comdlg32.dll 0x762B0000 0x4A000
userenv.dll 0x76920000 0xC4000
psapi.dll 0x76B70000 0xB000
secur32.dll 0x76F50000 0x13000
user32.dll 0x77380000 0x92000
comctl32.dll 0x77420000 0x103000
comctl32.dll 0x77530000 0x97000
version.dll 0x77B90000 0x8000
msvcrt.dll 0x77BA0000 0x5A000
gdi32.dll 0x77C00000 0x48000
rpcrt4.dll 0x77C50000 0x9F000
shlwapi.dll 0x77DA0000 0x52000
kernel32.dll 0x77E40000 0x102000
advapi32.dll 0x77F50000 0x9C000
msvcr71.dll 0x7C340000 0x56000
ntdll.dll 0x7C800000 0xC0000
shell32.dll 0x7C8D0000 0x803000
hythr.dll 0x84000000 0x407000
hysig.dll 0x84500000 0x6000
hyprt.dll 0x84550000 0x18000
hyzlib.dll 0x84600000 0x13000
hytext.dll 0x84650000 0x6000
vmi.dll 0x84700000 0x6000
hyluni.dll 0x84750000 0x23000
hyarchive.dll 0x84800000 0xD000
hynio.dll 0x84850000 0x6000
gc_gen.dll 0x85500000 0x2C000
harmonyvm.dll 0x85600000 0x424000
zlib1.dll 0x86100000 0x13000
em.dll 0x86200000 0x40000
jitrino.dll 0x86300000 0x410000
odbc32.dll 0x86800000 0x3D000
dbghelp.dll 0x86900000 0xA8000
icuuc34.dll 0x87030000 0xC8000
icudt34.dll 0x87100000 0x870000
Hopefully, my script will work OOB. If there any problems, we will try
to reiterate the moving one more time on another addresses. Note that
relocating of MS Windows system libraries is the challenge (and one
could consider this unfair) - since Windows will try to fight you :)
Thanks,
Aleksey Shipilev
Intel Enterprise Solutions Software Division
--
http://xiao-feng.blogspot.com
