Re: RFR(M)(round 2): 8215902: Add support for SoftFloat-3e library

[David Holmes]

Hi Jakub,

On 25/01/2019 9:17 am, Jakub Vaněk wrote:

Hi Magnus,

thanks for the review!

I haven't received a review for the hotspot source changes yet, so I
will have to wait.


Not an expert on the details of the FP code but the wrapper layer 
appears okay to me.


One nit with the test is that we don't name tests using bug numbers any 
more, so please rename the test to something more descriptive ... 
perhaps FloatPrecisionTest.java ?


Also on the test, given this wraps a number of FP functions does the 
test need to be more elaborate to ensure they have all been covered?


Thanks,
David
-


Regards,

Jakub

On 2019-01-23 at 13:55 +0100, Magnus Ihse Bursie wrote:

Hi Jakub,

On 2019-01-15 17:31, Jakub Vaněk wrote:

Hi Magnus and Erik,

I have added the link to the repository to README and I have
removed
the link to the mailing list thread. I have also recreated the
GitHub
repository. Now it is a fork of the mentioned repository with two
extra
commits containing README and the build scripts.

New webrev URL:
http://cr.openjdk.java.net/~jakvanek/8215902/webrev.04/
Bug: https://bugs.openjdk.java.net/browse/JDK-8215902


Sorry for the late reply.

This looks very good! Thank you for fixing this, including rebasing
the
github repo.

I'm not sure if you've gotten reviews from the hotspot team for the
hotspot source changes, but from a build perspective, this is good to
go.

/Magnus


Regards,

Jakub

On 2019-01-15 at 15:05 +0100, Magnus Ihse Bursie wrote:

On 2018-12-25 16:19, Jakub Vaněk wrote:

Hi,

please review this webrev. It is a successor of the softfloat-3
[patch]
thread (first email





http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-November/031311.html

)

Changes since the last patch (v6):

- renamed --with-softloat* to --with-sflt* (it is more compact
and
it
 corresponds to the old --with-sflt-lib=... option)

- license is now obtained via --with-sflt-license switch (so it
is
not
 included in OpenJDK source tree)

- updated documentation (slight rewording, added the license
option)

- checks for default --with/--without behavior are in place
again
 (I forgot them when I changed the way the library is
detected)

- added a simple testcase - I found a disrepancy between
softfloat
and
 system function behavior. When a float with bits 0x003F
is
 added to 0x0001, the correct result is 0x0040, but
the
 default software floating point implementation returns
0x.
 However I'm not sure where to put this test - now it is in
 test/hotspot/jtreg/compiler/floatingpoint.

- comments in code refer to CR 6757269 and newly JDK-8215902
too.

I have created a repository with SoftFloat-3e with build
configuration
specifically for OpenJDK on armel:
https://github.com/ev3dev-lang-java/softfloat-openjdk

I can add a link to it to the documentation.

Bug: https://bugs.openjdk.java.net/browse/JDK-8215902
Webrev: http://cr.openjdk.java.net/~jakvanek/8215902/webrev.02/


Hi Jakub,

In general this looks good.

Some comments:

I agree with Erik that you can add a link to your github project;
compiling SoftFloat is outside the scope of the OpenJDK build
instructions, but it can sure be helpful to lower the bar for
users
wanting to do that. Just one question: any particular reason you
didn't
create your github repo by forking the official
https://github.com/ucb-bar/berkeley-softfloat-3? That way, it
would
have
been easy for users to see that you were not adding any malicious
or
suspicious code to the original SoftFloat distribution.

On the other hand, I think the link to





http://mail.openjdk.java.net/pipermail/aarch32-port-dev/2016-November/000611.html
   
is unnecessary and just creates clutter in the documentation.

Please
remove it.

/Magnus

CI build:


https://ci.adoptopenjdk.net/view/ev3dev/job/openjdk12_build_ev3_linux/67/


Cheers,

Jakub

Re: Fail to cross compiling jdk12 for mips64el-linux-gnu target

[Leslie Zhai]

Hi Erik,

Thanks for your kind response!

在 2019/1/25 上午1:19, Erik Joelsson 写道:

Hello,

I agree with the conclusion here, there is certainly a bug in the 
build logic. When creating the "Build JDK" during a cross compilation 
build, we are, as Magnus said, cutting a few corners for efficiency. 
One of those corners is assuming all the java classes are the same and 
can simply be reused. It has been shown here that that assumption is 
not always correct.


I think the most reasonable solution is to assume all of java.base 
needs to be built specifically for the "Build JDK". I think this can 
be reasonably easy to implement (compared to the existing Build JDK 
logic, which is arguably rather hairy).


Magnus' suggested workaround should work fine for now however.


Failed to make images 
https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024765.html


But please point out my fault, thanks a lot!





Filed https://bugs.openjdk.java.net/browse/JDK-8217739


Thanks for your bug report!

Regards,

Leslie Zhai





/Erik

On 2019-01-24 05:30, Magnus Ihse Bursie wrote:

On 2019-01-24 13:22, Leslie Zhai wrote:

Hi Magnus,

Thanks for your kind response!

在 2019/1/24 下午8:05, Magnus Ihse Bursie 写道:



On 2019-01-24 05:45, David Holmes wrote:

On 24/01/2019 1:51 pm, Ao Qi wrote:

Hi David,

On Thu, Jan 24, 2019 at 10:47 AM David Holmes 
 wrote:


Hi Leslie,

I'm not Erik :) however 

On 24/01/2019 12:28 pm, Leslie Zhai wrote:

Hi Erik,

Because the flags' macro of MIPS is different from other 
targets[1], it

will effect the `UnixConstants` of
src/java.base/unix/classes/sun/nio/fs/UnixConstants.java.template, 
for
example, PREFIX_O_APPEND, so the `Flags` in `FileChannel` is 
different

from other targets.  Then failed[2] to cross compiling jdk12 for
mips64el-linux-gnu target by following the document[3].


It's been quite a while since we built for a platform where the 
flags
had different values, but it is supposed to work. I suspect that 
the way
we build now with the module system may not be quite correct in 
this
regard - but that is just supposition on my part. Because 
compile-time
constants are stored directly into class files that use them, we 
have to
generate UnixConstants.java before compiling any class that 
needs to use
it, then we must compile all classes that do use it. Those 
classes must

only be executed in conjunction with a tool executing on the target
platform (javac, jmod etc). My suspicion is that we may be 
executing a

tool on the build platform using the newly compiled classes for the
target platform - and that would be wrong.



Yes. Some variables in UnixConstants.java is defined according to 
the

target (fcntl.h of mips in this case), but would be executed on host
(x86 in this case) during "make images". These variables of ppc64le,
aarch64 and s390x are the same as x86, but the ones of mips are not.
However, mips is not an official supported target and I did not find
other targets have similar issue, I am not sure if this is a bug 
or if

this is the right mailing list :)


This is the right mailing list to discuss. :)

So I went back to find when this happened in the past and I 
mis-remembered how we solved it. We previously had a problem with 
the Oracle Java SE Embedded PPC port where some Linux PPC-e500v2 
platforms defined a different value for the O_NOFOLLOW and 
O_DIRECT flags. The solution then was to commit a platform 
specific version of UnixConstants.java and change the build system 
to only use the template file if there was not a pre-existing 
.java file. But that platform and the SE Embedded support was all 
removed. (SocketOptions.java had a similar problem.) Further that 
build logic is completely different to what we had back then (JDK 
6/7).


This is definitely a bug in our build logic IMHO as we 
specifically use the build platform c-pre-processor to process the 
template file based on the target header files, but then use the 
resulting class when running tools on the build platform. 
I agree with David. This is a bug in the build system; arguably 
even a regression.


This was not something that happened in 6/7 or even 8 and I think 
the module system tools are where the issue now lies.


I think Erik and/or Magnus will have to give advice on how this 
may be properly fixed. I can't see any simple solution.

Unfortunately, I also agree that there is no simple solution. :-(

The proper way to handle this is to generate two versions of 
UnixConstants and friends, one for the build platform, and one for 
the target platform. But as far as I know, there is no simple way 
to handle such complications in our current structure. Erik knows 
more about this, he was the one who designed the current solution 
for handling cross-compilation in the modularized world.


However, I can at least help you with a relatively simple 
workaround. First some background: When we cross-compile, we need 
not only a Boot JDK (of version current N-1) running on the build 

Re: Fail to cross compiling jdk12 for mips64el-linux-gnu target

[Leslie Zhai]

Hi Magnus,

Failed to make images:

$ make images CONF=mips
Building target 'images' in configuration 
'linux-mips64el-normal-server-fastdebug'
GenerateLinkOptData.gmk:61: recipe for target 
'/home/loongson/zhaixiang/jdk12-mips/build/linux-mips64el-normal-server-fastdebug/support/link_opt/classlist' 
failed
make[3]: *** 
[/home/loongson/zhaixiang/jdk12-mips/build/linux-mips64el-normal-server-fastdebug/support/link_opt/classlist] 
Error 1
make[3]: *** Deleting file 
'/home/loongson/zhaixiang/jdk12-mips/build/linux-mips64el-normal-server-fastdebug/support/link_opt/classlist'

make/Main.gmk:441: recipe for target 'generate-link-opt-data' failed
make[2]: *** [generate-link-opt-data] Error 2

ERROR: Build failed for target 'images' in configuration 
'linux-mips64el-normal-server-fastdebug' (exit code 2)


=== Make failed targets repeated here ===
GenerateLinkOptData.gmk:61: recipe for target 
'/home/loongson/zhaixiang/jdk12-mips/build/linux-mips64el-normal-server-fastdebug/support/link_opt/classlist' 
failed

make/Main.gmk:441: recipe for target 'generate-link-opt-data' failed
=== End of repeated output ===

Hint: Try searching the build log for the name of the first failed target.
Hint: See doc/building.html#troubleshooting for assistance.

/home/loongson/zhaixiang/jdk12-mips/make/Init.gmk:300: recipe for target 
'main' failed

make[1]: *** [main] Error 2
/home/loongson/zhaixiang/jdk12-mips/make/Init.gmk:186: recipe for target 
'images' failed

make: *** [images] Error 2

- 8<  8<  8<  8<  8<  8< ---

My cross-compile configure option:

CC=mips64el-linux-gnuabi64-gcc CXX=mips64el-linux-gnuabi64-g++ sh 
./configure \

  --openjdk-target=mips64el-linux-gnuabi64 \
  --with-sysroot=/chroots/mips64el_stretch \
  --with-toolchain-path=/chroots/mips64el_stretch \
  --with-boot-jdk=/home/loongson/aoqi/jdk-11.0.1 \
  --disable-warnings-as-errors \
--with-freetype-lib=/chroots/mips64el_stretch/usr/lib/mips64el-linux-gnuabi64 
\

--with-freetype-include=/chroots/mips64el_stretch/usr/include/freetype2 \
  --with-debug-level=fastdebug \
  --with-build-jdk=build/buildjdk/jdk

- 8<  8<  8<  8<  8<  8< ---

buildjdk is OK:

$ sh configure --with-conf-name=buildjdk 
--with-boot-jdk=/home/loongson/aoqi/jdk-11.0.1 --disable-warnings-as-errors


$ make jdk CONF=buildjdk

Building target 'jdk' in configuration 'buildjdk'
Finished building target 'jdk' in configuration 'buildjdk'

Regards,

Leslie Zhai


在 2019/1/24 下午9:30, Magnus Ihse Bursie 写道:

On 2019-01-24 13:22, Leslie Zhai wrote:

Hi Magnus,

Thanks for your kind response!

在 2019/1/24 下午8:05, Magnus Ihse Bursie 写道:



On 2019-01-24 05:45, David Holmes wrote:

On 24/01/2019 1:51 pm, Ao Qi wrote:

Hi David,

On Thu, Jan 24, 2019 at 10:47 AM David Holmes 
 wrote:


Hi Leslie,

I'm not Erik :) however 

On 24/01/2019 12:28 pm, Leslie Zhai wrote:

Hi Erik,

Because the flags' macro of MIPS is different from other 
targets[1], it

will effect the `UnixConstants` of
src/java.base/unix/classes/sun/nio/fs/UnixConstants.java.template, 
for
example, PREFIX_O_APPEND, so the `Flags` in `FileChannel` is 
different

from other targets.  Then failed[2] to cross compiling jdk12 for
mips64el-linux-gnu target by following the document[3].


It's been quite a while since we built for a platform where the 
flags
had different values, but it is supposed to work. I suspect that 
the way

we build now with the module system may not be quite correct in this
regard - but that is just supposition on my part. Because 
compile-time
constants are stored directly into class files that use them, we 
have to
generate UnixConstants.java before compiling any class that needs 
to use
it, then we must compile all classes that do use it. Those 
classes must

only be executed in conjunction with a tool executing on the target
platform (javac, jmod etc). My suspicion is that we may be 
executing a

tool on the build platform using the newly compiled classes for the
target platform - and that would be wrong.



Yes. Some variables in UnixConstants.java is defined according to the
target (fcntl.h of mips in this case), but would be executed on host
(x86 in this case) during "make images". These variables of ppc64le,
aarch64 and s390x are the same as x86, but the ones of mips are not.
However, mips is not an official supported target and I did not find
other targets have similar issue, I am not sure if this is a bug 
or if

this is the right mailing list :)


This is the right mailing list to discuss. :)

So I went back to find when this happened in the past and I 
mis-remembered how we solved it. We previously had a problem with 
the Oracle Java SE Embedded PPC port where some Linux PPC-e500v2 
platforms defined a different value for the O_NOFOLLOW and O_DIRECT 
flags. The solution then was to commit a platform specific version 
of UnixConstants.java and change the build system to only use the 
template file if there was 

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Martin Buchholz]

On Thu, Jan 24, 2019 at 2:28 PM Erik Joelsson  wrote:

> > Do the constituent object files have debugging information?  Sub-second
> > processing times for ~350 MiB of output are rather impressive.
>
> Ah! That must be it. I just tried with --with-native-debug-symbols=none
> and then I get comparable link times.

Yup!
Native debug symbols - more trouble than they're worth.

Re: RFR(M)(round 2): 8215902: Add support for SoftFloat-3e library

[Jakub Vaněk]

Hi Magnus,

thanks for the review!

I haven't received a review for the hotspot source changes yet, so I
will have to wait.

Regards,

Jakub

On 2019-01-23 at 13:55 +0100, Magnus Ihse Bursie wrote:
> Hi Jakub,
> 
> On 2019-01-15 17:31, Jakub Vaněk wrote:
> > Hi Magnus and Erik,
> > 
> > I have added the link to the repository to README and I have
> > removed
> > the link to the mailing list thread. I have also recreated the
> > GitHub
> > repository. Now it is a fork of the mentioned repository with two
> > extra
> > commits containing README and the build scripts.
> > 
> > New webrev URL: 
> > http://cr.openjdk.java.net/~jakvanek/8215902/webrev.04/
> > Bug: https://bugs.openjdk.java.net/browse/JDK-8215902
> 
> Sorry for the late reply.
> 
> This looks very good! Thank you for fixing this, including rebasing
> the 
> github repo.
> 
> I'm not sure if you've gotten reviews from the hotspot team for the 
> hotspot source changes, but from a build perspective, this is good to
> go.
> 
> /Magnus
> > 
> > Regards,
> > 
> > Jakub
> > 
> > On 2019-01-15 at 15:05 +0100, Magnus Ihse Bursie wrote:
> > > On 2018-12-25 16:19, Jakub Vaněk wrote:
> > > > Hi,
> > > > 
> > > > please review this webrev. It is a successor of the softfloat-3
> > > > [patch]
> > > > thread (first email
> > > > 
> > 
> > 
http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-November/031311.html
> > > > )
> > > > 
> > > > Changes since the last patch (v6):
> > > > 
> > > > - renamed --with-softloat* to --with-sflt* (it is more compact
> > > > and
> > > > it
> > > > corresponds to the old --with-sflt-lib=... option)
> > > > 
> > > > - license is now obtained via --with-sflt-license switch (so it
> > > > is
> > > > not
> > > > included in OpenJDK source tree)
> > > > 
> > > > - updated documentation (slight rewording, added the license
> > > > option)
> > > > 
> > > > - checks for default --with/--without behavior are in place
> > > > again
> > > > (I forgot them when I changed the way the library is
> > > > detected)
> > > > 
> > > > - added a simple testcase - I found a disrepancy between
> > > > softfloat
> > > > and
> > > > system function behavior. When a float with bits 0x003F
> > > > is
> > > > added to 0x0001, the correct result is 0x0040, but
> > > > the
> > > > default software floating point implementation returns
> > > > 0x.
> > > > However I'm not sure where to put this test - now it is in
> > > > test/hotspot/jtreg/compiler/floatingpoint.
> > > > 
> > > > - comments in code refer to CR 6757269 and newly JDK-8215902
> > > > too.
> > > > 
> > > > I have created a repository with SoftFloat-3e with build
> > > > configuration
> > > > specifically for OpenJDK on armel:
> > > > https://github.com/ev3dev-lang-java/softfloat-openjdk
> > > > 
> > > > I can add a link to it to the documentation.
> > > > 
> > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8215902
> > > > Webrev: http://cr.openjdk.java.net/~jakvanek/8215902/webrev.02/
> > > 
> > > Hi Jakub,
> > > 
> > > In general this looks good.
> > > 
> > > Some comments:
> > > 
> > > I agree with Erik that you can add a link to your github project;
> > > compiling SoftFloat is outside the scope of the OpenJDK build
> > > instructions, but it can sure be helpful to lower the bar for
> > > users
> > > wanting to do that. Just one question: any particular reason you
> > > didn't
> > > create your github repo by forking the official
> > > https://github.com/ucb-bar/berkeley-softfloat-3? That way, it
> > > would
> > > have
> > > been easy for users to see that you were not adding any malicious
> > > or
> > > suspicious code to the original SoftFloat distribution.
> > > 
> > > On the other hand, I think the link to
> > > 
> > 
> > 
http://mail.openjdk.java.net/pipermail/aarch32-port-dev/2016-November/000611.html
> > >   
> > > is unnecessary and just creates clutter in the documentation.
> > > Please
> > > remove it.
> > > 
> > > /Magnus
> > > > CI build:
> > > > 
https://ci.adoptopenjdk.net/view/ev3dev/job/openjdk12_build_ev3_linux/67/
> > > > 
> > > > Cheers,
> > > > 
> > > > Jakub
> > > > 
> 
> 

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Erik Joelsson]

On 2019-01-24 14:17, Florian Weimer wrote:

* Martin Buchholz:


Here's an experiment using the 3 competing open source linkers to link
hotspot.  This confirms that lld is faster than gold is faster than
bfd, but is the one second saving worth the engineering effort?

  $ (BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for
linker in bfd gold lld; do echo --- $linker ---; time /usr/bin/g++
-fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs -Wl,-z,noexecstack
-Wl,-O1 -Wl,-z,relro -m64 -static-libstdc++ -static-libgcc -shared
-m64 -Wl,-version-script=$BUILDDIR/hotspot/variant-server/libjvm/mapfile
-Wl,-soname=libjvm.so -o
$BUILDDIR/support/modules_libs/java.base/server/libjvm.so
@$BUILDDIR/hotspot/variant-server/libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt
-lm -ldl -lpthread; done)
--- bfd ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  1.31s user 0.36s system 99% cpu 1.669 total
--- gold ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  0.42s user 0.11s system 99% cpu 0.537 total
--- lld ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  0.25s user 0.20s system 145% cpu 0.310 total

Do the constituent object files have debugging information?  Sub-second
processing times for ~350 MiB of output are rather impressive.


Ah! That must be it. I just tried with --with-native-debug-symbols=none 
and then I get comparable link times.


/Erik


On a Sandybridge-era Xeon (probably without SSD storage), I see a change
from 13 seconds to 8 seconds (BFD to gold).  This is the first libjvm.so
link from “make images”, and the output includes (apparently full)
debugging information.

Thanks,
Florian

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Erik Joelsson]

Hello,

On 2019-01-24 13:18, Martin Buchholz wrote:

I just copied the command out of
hotspot/variant-server/libjvm/objs/BUILD_LIBJVM_link.cmdline
and lightly edited it.


I took the .cmdline verbatim and then just added -fuse-ld=gold, so 
should be the same. I'm running on Ubuntu 18.04 using a devkit produced 
by the makefiles in make/devkit. I'm running SSDs and AFAIK they are 
working as they should. When running the command using /usr/bin/time, I 
get 99% cpu. If I instead try the native toolchain on the system, the 
times are better, but still miles from yours:


With devkit, default linker: 23s
With devkit, gold: 14.2s
Native compiler, default linker: 12.5s
Native compiler, gold: 8.1s

The machine has 2 cpus identifying as "Intel(R) Xeon(R) CPU E5-2665 0 @ 
2.40GHz". These are certainly not known for single threaded performance, 
but I'm surprised by this difference.


Given our reality, I think you can understand why this change matters. 
Cutting off nearly 10s from incremental rebuilds is significant for 
developers when in an edit-build-test loop.


/Erik


On my old underpowered NUC at home I see slightly worse numbers (but
warmed up, files on SSD - are you I/O bound?).

(BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for ld in
bfd gold; do time /usr/bin/g++ -fuse-ld=$ld -Wl,--hash-style=both
-Wl,-z,defs -Wl,-z,noexecstack -Wl,-O1 -Wl,-z,relro -m64
-static-libstdc++ -static-libgcc -shared -m64
-Wl,-version-script=$BUILDDIR/hotspot/variant-server/libjvm/mapfile
-Wl,-soname=libjvm.so -o
$BUILDDIR/support/modules_libs/java.base/server/libjvm.so
@$BUILDDIR/hotspot/variant-server/libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt
-lm -ldl -lpthread; done)
/usr/bin/g++ -fuse-ld=$ld -Wl,--hash-style=both -Wl,-z,defs
-Wl,-z,noexecstac  2.28s user 0.33s system 99% cpu 2.621 total
/usr/bin/g++ -fuse-ld=$ld -Wl,--hash-style=both -Wl,-z,defs
-Wl,-z,noexecstac  0.65s user 0.09s system 98% cpu 0.747 total

On Thu, Jan 24, 2019 at 11:06 AM Erik Joelsson  wrote:

Are you actually linking libjvm.so in 1.3 seconds? A normal link time
for me using bfd is about 23 seconds while gold takes it to 14.2(+-0.2).
This is in line with what hotspot developers I have talked to also see
(and they have similar hardware).

My workstation has a few years on it, but surely machines haven't gotten
17 times faster? There must be something else at play here.

/Erik

On 2019-01-24 10:49, Martin Buchholz wrote:

Here's an experiment using the 3 competing open source linkers to link
hotspot.  This confirms that lld is faster than gold is faster than
bfd, but is the one second saving worth the engineering effort?

   $ (BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for
linker in bfd gold lld; do echo --- $linker ---; time /usr/bin/g++
-fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs -Wl,-z,noexecstack
-Wl,-O1 -Wl,-z,relro -m64 -static-libstdc++ -static-libgcc -shared
-m64 -Wl,-version-script=$BUILDDIR/hotspot/variant-server/libjvm/mapfile
-Wl,-soname=libjvm.so -o
$BUILDDIR/support/modules_libs/java.base/server/libjvm.so
@$BUILDDIR/hotspot/variant-server/libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt
-lm -ldl -lpthread; done)
--- bfd ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  1.31s user 0.36s system 99% cpu 1.669 total
--- gold ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  0.42s user 0.11s system 99% cpu 0.537 total
--- lld ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  0.25s user 0.20s system 145% cpu 0.310 total

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Florian Weimer]

* Martin Buchholz:

> Here's an experiment using the 3 competing open source linkers to link
> hotspot.  This confirms that lld is faster than gold is faster than
> bfd, but is the one second saving worth the engineering effort?
>
>  $ (BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for
> linker in bfd gold lld; do echo --- $linker ---; time /usr/bin/g++
> -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs -Wl,-z,noexecstack
> -Wl,-O1 -Wl,-z,relro -m64 -static-libstdc++ -static-libgcc -shared
> -m64 -Wl,-version-script=$BUILDDIR/hotspot/variant-server/libjvm/mapfile
> -Wl,-soname=libjvm.so -o
> $BUILDDIR/support/modules_libs/java.base/server/libjvm.so
> @$BUILDDIR/hotspot/variant-server/libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt
> -lm -ldl -lpthread; done)
> --- bfd ---
> /usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
> -Wl,-O1  -m6  1.31s user 0.36s system 99% cpu 1.669 total
> --- gold ---
> /usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
> -Wl,-O1  -m6  0.42s user 0.11s system 99% cpu 0.537 total
> --- lld ---
> /usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
> -Wl,-O1  -m6  0.25s user 0.20s system 145% cpu 0.310 total

Do the constituent object files have debugging information?  Sub-second
processing times for ~350 MiB of output are rather impressive.

On a Sandybridge-era Xeon (probably without SSD storage), I see a change
from 13 seconds to 8 seconds (BFD to gold).  This is the first libjvm.so
link from “make images”, and the output includes (apparently full)
debugging information.

Thanks,
Florian

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Martin Buchholz]

I just copied the command out of
hotspot/variant-server/libjvm/objs/BUILD_LIBJVM_link.cmdline
and lightly edited it.

On my old underpowered NUC at home I see slightly worse numbers (but
warmed up, files on SSD - are you I/O bound?).

(BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for ld in
bfd gold; do time /usr/bin/g++ -fuse-ld=$ld -Wl,--hash-style=both
-Wl,-z,defs -Wl,-z,noexecstack -Wl,-O1 -Wl,-z,relro -m64
-static-libstdc++ -static-libgcc -shared -m64
-Wl,-version-script=$BUILDDIR/hotspot/variant-server/libjvm/mapfile
-Wl,-soname=libjvm.so -o
$BUILDDIR/support/modules_libs/java.base/server/libjvm.so
@$BUILDDIR/hotspot/variant-server/libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt
-lm -ldl -lpthread; done)
/usr/bin/g++ -fuse-ld=$ld -Wl,--hash-style=both -Wl,-z,defs
-Wl,-z,noexecstac  2.28s user 0.33s system 99% cpu 2.621 total
/usr/bin/g++ -fuse-ld=$ld -Wl,--hash-style=both -Wl,-z,defs
-Wl,-z,noexecstac  0.65s user 0.09s system 98% cpu 0.747 total

On Thu, Jan 24, 2019 at 11:06 AM Erik Joelsson  wrote:
>
> Are you actually linking libjvm.so in 1.3 seconds? A normal link time
> for me using bfd is about 23 seconds while gold takes it to 14.2(+-0.2).
> This is in line with what hotspot developers I have talked to also see
> (and they have similar hardware).
>
> My workstation has a few years on it, but surely machines haven't gotten
> 17 times faster? There must be something else at play here.
>
> /Erik
>
> On 2019-01-24 10:49, Martin Buchholz wrote:
> > Here's an experiment using the 3 competing open source linkers to link
> > hotspot.  This confirms that lld is faster than gold is faster than
> > bfd, but is the one second saving worth the engineering effort?
> >
> >   $ (BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for
> > linker in bfd gold lld; do echo --- $linker ---; time /usr/bin/g++
> > -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs -Wl,-z,noexecstack
> > -Wl,-O1 -Wl,-z,relro -m64 -static-libstdc++ -static-libgcc -shared
> > -m64 -Wl,-version-script=$BUILDDIR/hotspot/variant-server/libjvm/mapfile
> > -Wl,-soname=libjvm.so -o
> > $BUILDDIR/support/modules_libs/java.base/server/libjvm.so
> > @$BUILDDIR/hotspot/variant-server/libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt
> > -lm -ldl -lpthread; done)
> > --- bfd ---
> > /usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
> > -Wl,-O1  -m6  1.31s user 0.36s system 99% cpu 1.669 total
> > --- gold ---
> > /usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
> > -Wl,-O1  -m6  0.42s user 0.11s system 99% cpu 0.537 total
> > --- lld ---
> > /usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
> > -Wl,-O1  -m6  0.25s user 0.20s system 145% cpu 0.310 total

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Erik Joelsson]

Are you actually linking libjvm.so in 1.3 seconds? A normal link time 
for me using bfd is about 23 seconds while gold takes it to 14.2(+-0.2). 
This is in line with what hotspot developers I have talked to also see 
(and they have similar hardware).


My workstation has a few years on it, but surely machines haven't gotten 
17 times faster? There must be something else at play here.


/Erik

On 2019-01-24 10:49, Martin Buchholz wrote:

Here's an experiment using the 3 competing open source linkers to link
hotspot.  This confirms that lld is faster than gold is faster than
bfd, but is the one second saving worth the engineering effort?

  $ (BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for
linker in bfd gold lld; do echo --- $linker ---; time /usr/bin/g++
-fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs -Wl,-z,noexecstack
-Wl,-O1 -Wl,-z,relro -m64 -static-libstdc++ -static-libgcc -shared
-m64 -Wl,-version-script=$BUILDDIR/hotspot/variant-server/libjvm/mapfile
-Wl,-soname=libjvm.so -o
$BUILDDIR/support/modules_libs/java.base/server/libjvm.so
@$BUILDDIR/hotspot/variant-server/libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt
-lm -ldl -lpthread; done)
--- bfd ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  1.31s user 0.36s system 99% cpu 1.669 total
--- gold ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  0.42s user 0.11s system 99% cpu 0.537 total
--- lld ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  0.25s user 0.20s system 145% cpu 0.310 total

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Martin Buchholz]

Here's an experiment using the 3 competing open source linkers to link
hotspot.  This confirms that lld is faster than gold is faster than
bfd, but is the one second saving worth the engineering effort?

 $ (BUILDDIR=$HOME/ws/jdk/build/linux-x86_64-server-release; for
linker in bfd gold lld; do echo --- $linker ---; time /usr/bin/g++
-fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs -Wl,-z,noexecstack
-Wl,-O1 -Wl,-z,relro -m64 -static-libstdc++ -static-libgcc -shared
-m64 -Wl,-version-script=$BUILDDIR/hotspot/variant-server/libjvm/mapfile
-Wl,-soname=libjvm.so -o
$BUILDDIR/support/modules_libs/java.base/server/libjvm.so
@$BUILDDIR/hotspot/variant-server/libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt
-lm -ldl -lpthread; done)
--- bfd ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  1.31s user 0.36s system 99% cpu 1.669 total
--- gold ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  0.42s user 0.11s system 99% cpu 0.537 total
--- lld ---
/usr/bin/g++ -fuse-ld=$linker -Wl,--hash-style=both -Wl,-z,defs
-Wl,-O1  -m6  0.25s user 0.20s system 145% cpu 0.310 total

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Andrew Haley]

On 1/24/19 1:25 PM, Aleksey Shipilev wrote:

> That'd be nice. Maybe we should check for the presence of ld.gold,
> though, and override LD when it is available? That would also allow
> the fallback when ld.gold is misbehaving, but setting LD forcefully.

Yes, we absolutely should check. I just built the latest buinutils, and
gold is not enabled by default. Apart from anything else, that means gold
won't get so much testing.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Magnus Ihse Bursie]

On 2019-01-24 18:43, Erik Joelsson wrote:


On 2019-01-24 06:04, Magnus Ihse Bursie wrote:

On 2019-01-24 14:45, Florian Weimer wrote:

* Magnus Ihse Bursie:


The default binutils linker used by gcc, the bfd linker, is slow. The
new replacement, gold, has been distributed alongside gcc for several
years now, and is a well mature, and much faster, replacement.

The gold linker is an optional component of binutils, not available in
all builds.  For example, binutils in Red Hat Enterprise Linux 7.6 does
not include gold on the ppc64le architecture.

The gold linker also supports a different set of features compared to
BFD ld, which may or may not be what you want.  But I think OpenJDK 
does

not use many tricky ELF features, so the differences probably do not
matter.

Is it possible to add -fuse-ld=gold to LDFLAGS externally, outside the
build system, so that the build system can use the gold linker if 
people

prefer it over BFD ld?

Or you could configure your binutils with --enable-gold=default, so 
that

it defaults to ld.gold, again not requiring any OpenJDK changes.
One of the driving forces behind this bug is the speed increase in 
gold by itself. You are correct that to achieve this, a solution 
outside the build system can be used.


However, the other driving force is the ability to enable incremental 
linking. The build system must know if we use gold, so that it knows 
that those command line options are available. So to resort to the 
solution of changing the environment would not enable that second part.


But I'm leaning more towards just enabling gold on x86_64 -- for 
other platforms, we might as well keep the good ol' bfd linker. Does 
that sound like a good solution to you?


I have been looking at this before (probably a couple of years ago 
now) and my take then was that this should be handled with a configure 
flag and be opt in (that jib would set for us). The default really 
should be to accept what the toolchain/admin/distribution has as 
default. If --enable-gold is given, configure needs to check for gold 
availability. I had to explicitly enable it in the devkit builds, so 
it's definitely not always present.


We could also just make sure our internal devkit uses gold by default, 
and leave explicit setting of this to simply adding a linker arg with 
configure.


By using an explicit configure argument to enable it, configure knows 
about it and we can enable incremental linking using that information. 
Incremental linking must be opt in though. We could also skip the 
--enable-gold flag and just do the checks if 
--enable-incremental-linking is set.


Last I experimented with it, incremental did not have a positive 
effect on the time to link hotspot, but gold did.
Ok. I'll retract my RFR for now, and come back with a way to set this 
conditionally using configure flags.


/Magnus


/Magnus




Thanks,
Florian

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Erik Joelsson]

On 2019-01-24 06:04, Magnus Ihse Bursie wrote:

On 2019-01-24 14:45, Florian Weimer wrote:

* Magnus Ihse Bursie:


The default binutils linker used by gcc, the bfd linker, is slow. The
new replacement, gold, has been distributed alongside gcc for several
years now, and is a well mature, and much faster, replacement.

The gold linker is an optional component of binutils, not available in
all builds.  For example, binutils in Red Hat Enterprise Linux 7.6 does
not include gold on the ppc64le architecture.

The gold linker also supports a different set of features compared to
BFD ld, which may or may not be what you want.  But I think OpenJDK does
not use many tricky ELF features, so the differences probably do not
matter.

Is it possible to add -fuse-ld=gold to LDFLAGS externally, outside the
build system, so that the build system can use the gold linker if people
prefer it over BFD ld?

Or you could configure your binutils with --enable-gold=default, so that
it defaults to ld.gold, again not requiring any OpenJDK changes.
One of the driving forces behind this bug is the speed increase in 
gold by itself. You are correct that to achieve this, a solution 
outside the build system can be used.


However, the other driving force is the ability to enable incremental 
linking. The build system must know if we use gold, so that it knows 
that those command line options are available. So to resort to the 
solution of changing the environment would not enable that second part.


But I'm leaning more towards just enabling gold on x86_64 -- for other 
platforms, we might as well keep the good ol' bfd linker. Does that 
sound like a good solution to you?


I have been looking at this before (probably a couple of years ago now) 
and my take then was that this should be handled with a configure flag 
and be opt in (that jib would set for us). The default really should be 
to accept what the toolchain/admin/distribution has as default. If 
--enable-gold is given, configure needs to check for gold availability. 
I had to explicitly enable it in the devkit builds, so it's definitely 
not always present.


We could also just make sure our internal devkit uses gold by default, 
and leave explicit setting of this to simply adding a linker arg with 
configure.


By using an explicit configure argument to enable it, configure knows 
about it and we can enable incremental linking using that information. 
Incremental linking must be opt in though. We could also skip the 
--enable-gold flag and just do the checks if 
--enable-incremental-linking is set.


Last I experimented with it, incremental did not have a positive effect 
on the time to link hotspot, but gold did.


/Magnus




Thanks,
Florian

Re: RFR: JDK-8217733 Minor RunTest fixes

[Alexandre (Shura) Iline]

Looks good.

Shura

> On Jan 24, 2019, at 6:46 AM, Magnus Ihse Bursie 
>  wrote:
> 
> This patch covers some different minor cleanups that I accumulated after 
> working on RunTest:
>  * Do not override JDK_IMAGE_DIR for jcov testing; instead introduce 
> JDK_UNDER_TEST (to be really explicit about which JDK we're testing).
>  * Remove unconditional $(info) printout on Windows
>  * Group JCov variable setup with other variable setups.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8217733
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8217733-minor-RunTest-fixes/webrev.01
> 
> /Magnus

Re: RFR: JDK-8217733 Minor RunTest fixes

[Erik Joelsson]

Looks good.

/Erik

On 2019-01-24 06:46, Magnus Ihse Bursie wrote:
This patch covers some different minor cleanups that I accumulated 
after working on RunTest:
 * Do not override JDK_IMAGE_DIR for jcov testing; instead introduce 
JDK_UNDER_TEST (to be really explicit about which JDK we're testing).

 * Remove unconditional $(info) printout on Windows
 * Group JCov variable setup with other variable setups.

Bug: https://bugs.openjdk.java.net/browse/JDK-8217733
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8217733-minor-RunTest-fixes/webrev.01


/Magnus

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Martin Buchholz]

On Thu, Jan 24, 2019 at 7:46 AM Magnus Ihse Bursie
 wrote:

> So, we already tacitly assume a specific linker with the gcc toolchain; we 
> have just not made that check explicitly.

gcc  runs on almost every system, but it's harder to replace the
compiler than the linker, so gcc ends up being used with many other OS
default linkers.
Same for clang.
Meanwhile, we have 3 competing open source linkers: bfd gold lld, so
linker availability is especially fluid these days.
Incremental linking will only be of interest to openjdk developers
(and then probably only hotspot developers) for non-release builds, so
should be opt-in.

Re: RFR: JDK-8217730 Split up MakeBase.gmk

[Erik Joelsson]

Looks good.

/Erik

On 2019-01-24 06:24, Magnus Ihse Bursie wrote:
The file MakeBase.gmk, which is included by virtually all other make 
files in the build system, has become gradually larger and larger. It 
is now one of the longest files in the build system, with little to no 
internal structure.


I propose to split it in four parts:

* Utils.gmk -- here all the simple make functions go; stuff that 
basically do not interact with the rest of the world, but just 
provides "high-level" macros to make.


* CopyFiles.gmk -- here the SetupCopyFiles function goes, similar to 
how we treat all other SetupFoo functions.


* MakeIO.gmk -- here reading and writing to files from make goes. This 
is mostly due to the long expression of ListPathsSafely on GNU Make 
prior to 4.0.


*MakeBase.gmk -- functions now remaining here belong to either of 
these groups:
 -- it is some kind of core functionality provided by the build 
system, like ExecuteWithLog or NamedParamsMacroTemplate.

 -- it interacts (messily) with the environment
 -- it does something else that is tricky, or just "misc"

It is of course possible to break down the remaining MakeBase.gmk even 
further, but it's so much more manageable now, and there are no clear 
ways of splitting it further without getting into very small pieces 
(like logging), so I think I'll stop here, for now.


To retain backwards compatibility with files that include MakeBase.gmk 
and expect all functionality, MakeBase.gmk includes the new files. 
It's a future question if we should keep it that way, or put the 
requirement of those includes on the files needing the functionality.


Bug: https://bugs.openjdk.java.net/browse/JDK-8217730
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8217730-split-up-MakeBase/webrev.01


/Magnus

Re: Fail to cross compiling jdk12 for mips64el-linux-gnu target

[Erik Joelsson]

Hello,

I agree with the conclusion here, there is certainly a bug in the build 
logic. When creating the "Build JDK" during a cross compilation build, 
we are, as Magnus said, cutting a few corners for efficiency. One of 
those corners is assuming all the java classes are the same and can 
simply be reused. It has been shown here that that assumption is not 
always correct.


I think the most reasonable solution is to assume all of java.base needs 
to be built specifically for the "Build JDK". I think this can be 
reasonably easy to implement (compared to the existing Build JDK logic, 
which is arguably rather hairy).


Magnus' suggested workaround should work fine for now however.

Filed https://bugs.openjdk.java.net/browse/JDK-8217739

/Erik

On 2019-01-24 05:30, Magnus Ihse Bursie wrote:

On 2019-01-24 13:22, Leslie Zhai wrote:

Hi Magnus,

Thanks for your kind response!

在 2019/1/24 下午8:05, Magnus Ihse Bursie 写道:



On 2019-01-24 05:45, David Holmes wrote:

On 24/01/2019 1:51 pm, Ao Qi wrote:

Hi David,

On Thu, Jan 24, 2019 at 10:47 AM David Holmes 
 wrote:


Hi Leslie,

I'm not Erik :) however 

On 24/01/2019 12:28 pm, Leslie Zhai wrote:

Hi Erik,

Because the flags' macro of MIPS is different from other 
targets[1], it

will effect the `UnixConstants` of
src/java.base/unix/classes/sun/nio/fs/UnixConstants.java.template, 
for
example, PREFIX_O_APPEND, so the `Flags` in `FileChannel` is 
different

from other targets.  Then failed[2] to cross compiling jdk12 for
mips64el-linux-gnu target by following the document[3].


It's been quite a while since we built for a platform where the 
flags
had different values, but it is supposed to work. I suspect that 
the way

we build now with the module system may not be quite correct in this
regard - but that is just supposition on my part. Because 
compile-time
constants are stored directly into class files that use them, we 
have to
generate UnixConstants.java before compiling any class that needs 
to use
it, then we must compile all classes that do use it. Those 
classes must

only be executed in conjunction with a tool executing on the target
platform (javac, jmod etc). My suspicion is that we may be 
executing a

tool on the build platform using the newly compiled classes for the
target platform - and that would be wrong.



Yes. Some variables in UnixConstants.java is defined according to the
target (fcntl.h of mips in this case), but would be executed on host
(x86 in this case) during "make images". These variables of ppc64le,
aarch64 and s390x are the same as x86, but the ones of mips are not.
However, mips is not an official supported target and I did not find
other targets have similar issue, I am not sure if this is a bug 
or if

this is the right mailing list :)


This is the right mailing list to discuss. :)

So I went back to find when this happened in the past and I 
mis-remembered how we solved it. We previously had a problem with 
the Oracle Java SE Embedded PPC port where some Linux PPC-e500v2 
platforms defined a different value for the O_NOFOLLOW and O_DIRECT 
flags. The solution then was to commit a platform specific version 
of UnixConstants.java and change the build system to only use the 
template file if there was not a pre-existing .java file. But that 
platform and the SE Embedded support was all removed. 
(SocketOptions.java had a similar problem.) Further that build 
logic is completely different to what we had back then (JDK 6/7).


This is definitely a bug in our build logic IMHO as we specifically 
use the build platform c-pre-processor to process the template file 
based on the target header files, but then use the resulting class 
when running tools on the build platform. 
I agree with David. This is a bug in the build system; arguably even 
a regression.


This was not something that happened in 6/7 or even 8 and I think 
the module system tools are where the issue now lies.


I think Erik and/or Magnus will have to give advice on how this may 
be properly fixed. I can't see any simple solution.

Unfortunately, I also agree that there is no simple solution. :-(

The proper way to handle this is to generate two versions of 
UnixConstants and friends, one for the build platform, and one for 
the target platform. But as far as I know, there is no simple way to 
handle such complications in our current structure. Erik knows more 
about this, he was the one who designed the current solution for 
handling cross-compilation in the modularized world.


However, I can at least help you with a relatively simple 
workaround. First some background: When we cross-compile, we need 
not only a Boot JDK (of version current N-1) running on the build 
host platform, but we also need a Build JDK, based on the current 
source code, running on the build host. (This is for running 
jmod/jlink; it needs to be up to date). If we have no Build JDK 
present, we start by creating one ourselves. This is more or less 
the same output as running a 

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Magnus Ihse Bursie]

I agree that the relationship between compiler, linker and OS is problematic. 
In the OpenJDK build system, we have used the concept of "toolchain", by which 
we mean (amongst other things) a compiler and a linker working together. For 
some platforms it's a perfect fit (eg Visual Studio), for others like gcc it's 
so-so, for yet others (solstudio) it's really bad. 

So, we already tacitly assume a specific linker with the gcc toolchain; we have 
just not made that check explicitly. 

As I said in a previous email, part of the reason for this switch is to allow 
--incremental. So for that reason, we must know that the linker supports that 
option. 

/Magnus

> 24 jan. 2019 kl. 15:48 skrev Martin Buchholz :
> 
> Getting into the business of choosing the linker is big trouble.
> 
> The system default toolchain may have already chosen a linker.
> 
> BFD might be configured to have either bfd ld or gold.
> # Handle --enable-gold, --enable-ld.
> # --disable-gold [--enable-ld]
> # Build only ld.  Default option.
> # --enable-gold [--enable-ld]
> # Build both gold and ld.  Install gold as "ld.gold", install ld
> # as "ld.bfd" and "ld".
> # --enable-gold=default [--enable-ld]
> # Build both gold and ld.  Install gold as "ld.gold" and "ld",
> # install ld as "ld.bfd".
> # --enable-gold[=default] --disable-ld
> # Build only gold, which is then installed as both "ld.gold" and "ld".
> # --enable-gold --enable-ld=default
> # Build both gold (installed as "ld.gold") and ld (installed as "ld"
> # and ld.bfd).
> # In other words, ld is default
> 
> The compiler driver itself may have been configured to choose a linker.
> 
> The system administrator may have used update-alternatives to choose a linker.
> 
> A user might have configured openjdk to use -fuse-ld=... (we do this!)
> How do you resolve the conflict?
> 
> There's evidence that lld is even faster than gold.  The Internet says,
> """Liked linking 3x faster with gold? Link 10x faster with lld!"""
> so hardcoding gold might be a regression!
> 
> So ... -fuse-ld is a flag that is perfect for a local wrapper around
> configure that is __not__ part of openjdk (we do this!)

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Martin Buchholz]

Getting into the business of choosing the linker is big trouble.

The system default toolchain may have already chosen a linker.

BFD might be configured to have either bfd ld or gold.
# Handle --enable-gold, --enable-ld.
# --disable-gold [--enable-ld]
# Build only ld.  Default option.
# --enable-gold [--enable-ld]
# Build both gold and ld.  Install gold as "ld.gold", install ld
# as "ld.bfd" and "ld".
# --enable-gold=default [--enable-ld]
# Build both gold and ld.  Install gold as "ld.gold" and "ld",
# install ld as "ld.bfd".
# --enable-gold[=default] --disable-ld
# Build only gold, which is then installed as both "ld.gold" and "ld".
# --enable-gold --enable-ld=default
# Build both gold (installed as "ld.gold") and ld (installed as "ld"
# and ld.bfd).
# In other words, ld is default

The compiler driver itself may have been configured to choose a linker.

The system administrator may have used update-alternatives to choose a linker.

A user might have configured openjdk to use -fuse-ld=... (we do this!)
How do you resolve the conflict?

There's evidence that lld is even faster than gold.  The Internet says,
"""Liked linking 3x faster with gold? Link 10x faster with lld!"""
so hardcoding gold might be a regression!

So ... -fuse-ld is a flag that is perfect for a local wrapper around
configure that is __not__ part of openjdk (we do this!)

RFR: JDK-8217733 Minor RunTest fixes

[Magnus Ihse Bursie]

This patch covers some different minor cleanups that I accumulated after 
working on RunTest:
 * Do not override JDK_IMAGE_DIR for jcov testing; instead introduce 
JDK_UNDER_TEST (to be really explicit about which JDK we're testing).

 * Remove unconditional $(info) printout on Windows
 * Group JCov variable setup with other variable setups.

Bug: https://bugs.openjdk.java.net/browse/JDK-8217733
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8217733-minor-RunTest-fixes/webrev.01


/Magnus

RFR: JDK-8217730 Split up MakeBase.gmk

[Magnus Ihse Bursie]

The file MakeBase.gmk, which is included by virtually all other make 
files in the build system, has become gradually larger and larger. It is 
now one of the longest files in the build system, with little to no 
internal structure.


I propose to split it in four parts:

* Utils.gmk -- here all the simple make functions go; stuff that 
basically do not interact with the rest of the world, but just provides 
"high-level" macros to make.


* CopyFiles.gmk -- here the SetupCopyFiles function goes, similar to how 
we treat all other SetupFoo functions.


* MakeIO.gmk -- here reading and writing to files from make goes. This 
is mostly due to the long expression of ListPathsSafely on GNU Make 
prior to 4.0.


*MakeBase.gmk -- functions now remaining here belong to either of these 
groups:
 -- it is some kind of core functionality provided by the build system, 
like ExecuteWithLog or NamedParamsMacroTemplate.

 -- it interacts (messily) with the environment
 -- it does something else that is tricky, or just "misc"

It is of course possible to break down the remaining MakeBase.gmk even 
further, but it's so much more manageable now, and there are no clear 
ways of splitting it further without getting into very small pieces 
(like logging), so I think I'll stop here, for now.


To retain backwards compatibility with files that include MakeBase.gmk 
and expect all functionality, MakeBase.gmk includes the new files. It's 
a future question if we should keep it that way, or put the requirement 
of those includes on the files needing the functionality.


Bug: https://bugs.openjdk.java.net/browse/JDK-8217730
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8217730-split-up-MakeBase/webrev.01


/Magnus

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Magnus Ihse Bursie]

On 2019-01-24 14:45, Florian Weimer wrote:

* Magnus Ihse Bursie:


The default binutils linker used by gcc, the bfd linker, is slow. The
new replacement, gold, has been distributed alongside gcc for several
years now, and is a well mature, and much faster, replacement.

The gold linker is an optional component of binutils, not available in
all builds.  For example, binutils in Red Hat Enterprise Linux 7.6 does
not include gold on the ppc64le architecture.

The gold linker also supports a different set of features compared to
BFD ld, which may or may not be what you want.  But I think OpenJDK does
not use many tricky ELF features, so the differences probably do not
matter.

Is it possible to add -fuse-ld=gold to LDFLAGS externally, outside the
build system, so that the build system can use the gold linker if people
prefer it over BFD ld?

Or you could configure your binutils with --enable-gold=default, so that
it defaults to ld.gold, again not requiring any OpenJDK changes.
One of the driving forces behind this bug is the speed increase in gold 
by itself. You are correct that to achieve this, a solution outside the 
build system can be used.


However, the other driving force is the ability to enable incremental 
linking. The build system must know if we use gold, so that it knows 
that those command line options are available. So to resort to the 
solution of changing the environment would not enable that second part.


But I'm leaning more towards just enabling gold on x86_64 -- for other 
platforms, we might as well keep the good ol' bfd linker. Does that 
sound like a good solution to you?


/Magnus



Thanks,
Florian

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Florian Weimer]

* Magnus Ihse Bursie:

> The default binutils linker used by gcc, the bfd linker, is slow. The
> new replacement, gold, has been distributed alongside gcc for several
> years now, and is a well mature, and much faster, replacement.

The gold linker is an optional component of binutils, not available in
all builds.  For example, binutils in Red Hat Enterprise Linux 7.6 does
not include gold on the ppc64le architecture.

The gold linker also supports a different set of features compared to
BFD ld, which may or may not be what you want.  But I think OpenJDK does
not use many tricky ELF features, so the differences probably do not
matter.

Is it possible to add -fuse-ld=gold to LDFLAGS externally, outside the
build system, so that the build system can use the gold linker if people
prefer it over BFD ld?

Or you could configure your binutils with --enable-gold=default, so that
it defaults to ld.gold, again not requiring any OpenJDK changes.

Thanks,
Florian

Re: Fail to cross compiling jdk12 for mips64el-linux-gnu target

[Magnus Ihse Bursie]

On 2019-01-24 13:22, Leslie Zhai wrote:

Hi Magnus,

Thanks for your kind response!

在 2019/1/24 下午8:05, Magnus Ihse Bursie 写道:



On 2019-01-24 05:45, David Holmes wrote:

On 24/01/2019 1:51 pm, Ao Qi wrote:

Hi David,

On Thu, Jan 24, 2019 at 10:47 AM David Holmes 
 wrote:


Hi Leslie,

I'm not Erik :) however 

On 24/01/2019 12:28 pm, Leslie Zhai wrote:

Hi Erik,

Because the flags' macro of MIPS is different from other 
targets[1], it

will effect the `UnixConstants` of
src/java.base/unix/classes/sun/nio/fs/UnixConstants.java.template, 
for
example, PREFIX_O_APPEND, so the `Flags` in `FileChannel` is 
different

from other targets.  Then failed[2] to cross compiling jdk12 for
mips64el-linux-gnu target by following the document[3].


It's been quite a while since we built for a platform where the flags
had different values, but it is supposed to work. I suspect that 
the way

we build now with the module system may not be quite correct in this
regard - but that is just supposition on my part. Because 
compile-time
constants are stored directly into class files that use them, we 
have to
generate UnixConstants.java before compiling any class that needs 
to use
it, then we must compile all classes that do use it. Those classes 
must

only be executed in conjunction with a tool executing on the target
platform (javac, jmod etc). My suspicion is that we may be 
executing a

tool on the build platform using the newly compiled classes for the
target platform - and that would be wrong.



Yes. Some variables in UnixConstants.java is defined according to the
target (fcntl.h of mips in this case), but would be executed on host
(x86 in this case) during "make images". These variables of ppc64le,
aarch64 and s390x are the same as x86, but the ones of mips are not.
However, mips is not an official supported target and I did not find
other targets have similar issue, I am not sure if this is a bug or if
this is the right mailing list :)


This is the right mailing list to discuss. :)

So I went back to find when this happened in the past and I 
mis-remembered how we solved it. We previously had a problem with 
the Oracle Java SE Embedded PPC port where some Linux PPC-e500v2 
platforms defined a different value for the O_NOFOLLOW and O_DIRECT 
flags. The solution then was to commit a platform specific version 
of UnixConstants.java and change the build system to only use the 
template file if there was not a pre-existing .java file. But that 
platform and the SE Embedded support was all removed. 
(SocketOptions.java had a similar problem.) Further that build logic 
is completely different to what we had back then (JDK 6/7).


This is definitely a bug in our build logic IMHO as we specifically 
use the build platform c-pre-processor to process the template file 
based on the target header files, but then use the resulting class 
when running tools on the build platform. 
I agree with David. This is a bug in the build system; arguably even 
a regression.


This was not something that happened in 6/7 or even 8 and I think 
the module system tools are where the issue now lies.


I think Erik and/or Magnus will have to give advice on how this may 
be properly fixed. I can't see any simple solution.

Unfortunately, I also agree that there is no simple solution. :-(

The proper way to handle this is to generate two versions of 
UnixConstants and friends, one for the build platform, and one for 
the target platform. But as far as I know, there is no simple way to 
handle such complications in our current structure. Erik knows more 
about this, he was the one who designed the current solution for 
handling cross-compilation in the modularized world.


However, I can at least help you with a relatively simple workaround. 
First some background: When we cross-compile, we need not only a Boot 
JDK (of version current N-1) running on the build host platform, but 
we also need a Build JDK, based on the current source code, running 
on the build host. (This is for running jmod/jlink; it needs to be up 
to date). If we have no Build JDK present, we start by creating one 
ourselves. This is more or less the same output as running a normal, 
non-cross compiled, build on the build host. (But Erik found some 
ways to cut a few corners to save time.) Your problem is that the 
Build JDK is broken, since UnixConstants has the wrong values.

-
But you can supply your own working Build JDK to configure!

So, your workflow for cross-compiling to MIPS should be:
1) create a Build JDK, e.g. like this: "bash configure 
--with-conf-name=buildjdk && make jdk CONF=buildjdk"
2) create your cross-compilation JDK like this: "bash configure 
--openjdk-target=mipsel-unknown-linux-gnu 
--with-build-jdk=build/buildjdk/jdk"

... or something like that.
Then you can build your cross-compiled jdk with "make images CONF=mips".


I will try that, thanks a lot for your teaching!
Please let me know if it works. I wrote the command lines out of my 
head, so 

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Magnus Ihse Bursie]

On 2019-01-24 14:20, John Paul Adrian Glaubitz wrote:

On 1/24/19 2:11 PM, Magnus Ihse Bursie wrote:

The default binutils linker used by gcc, the bfd linker, is slow.
The new replacement, gold, has been distributed alongside gcc for
several years now, and is a well mature, and much faster, replacement.

I would recommend testing this change on all supported targets. Gold is
definitely not as mature on all targets as bfd in my experience. There
are still some cases where it can cause issues.

Good point.

I was mainly considering x86_64 here. It might be safest to limit the 
use of gold to this platform. From the developer point of view, it's the 
one that counts most.


But even if I change this to only use gold on x86_64, I'd still like to 
bump the binutils version globally. There's no good way to express 
different version requirements on different platforms for the same 
toolchain. Do you see a problem with that?


/Magnus




Adrian

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Aleksey Shipilev]

On 1/24/19 2:15 PM, Magnus Ihse Bursie wrote:
> I see now on Wikipedia that gold was added to binutils 2.19.[1] (I thought it 
> was older still...)
> We currently have this check:
> TOOLCHAIN_MINIMUM_LD_VERSION_gcc="2.18"
> 
> As part of this fix, I ought to bump it to 2.19, so we know for sure that 
> gold is included.

Not sure it would be compatible with all current build environments. I asked 
Red Hat people if this
would be problematic for them.

>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217723
>> Patch inline:
>> diff --git a/make/autoconf/flags-ldflags.m4 b/make/autoconf/flags-ldflags.m4
>> --- a/make/autoconf/flags-ldflags.m4
>> +++ b/make/autoconf/flags-ldflags.m4
>> @@ -70,7 +70,7 @@
>>  fi
>>
>>  # Add -z defs, to forbid undefined symbols in object files.
>> -    BASIC_LDFLAGS="$BASIC_LDFLAGS -Wl,-z,defs"
>> +    BASIC_LDFLAGS="$BASIC_LDFLAGS -Wl,-z,defs -fuse-ld=gold"
>>
>>  BASIC_LDFLAGS_JVM_ONLY="-Wl,-O1 -Wl,-z,relro"

That'd be nice. Maybe we should check for the presence of ld.gold, though, and 
override LD when it
is available? That would also allow the fallback when ld.gold is misbehaving, 
but setting LD forcefully.

-Aleksey

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[John Paul Adrian Glaubitz]

On 1/24/19 2:11 PM, Magnus Ihse Bursie wrote:
> The default binutils linker used by gcc, the bfd linker, is slow.
> The new replacement, gold, has been distributed alongside gcc for
> several years now, and is a well mature, and much faster, replacement.

I would recommend testing this change on all supported targets. Gold is
definitely not as mature on all targets as bfd in my experience. There
are still some cases where it can cause issues.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Magnus Ihse Bursie]

I see now on Wikipedia that gold was added to binutils 2.19.[1] (I 
thought it was older still...)


We currently have this check:

TOOLCHAIN_MINIMUM_LD_VERSION_gcc="2.18"

As part of this fix, I ought to bump it to 2.19, so we know for sure 
that gold is included.


Volker, would that be a problem for you?

/Magnus
[1] https://en.wikipedia.org/wiki/Gold_(linker)

On 2019-01-24 14:11, Magnus Ihse Bursie wrote:
The default binutils linker used by gcc, the bfd linker, is slow. The 
new replacement, gold, has been distributed alongside gcc for several 
years now, and is a well mature, and much faster, replacement.


This issue is about replacing ld.bfd with ld.gold for our gcc 
toolchain. In general, this cuts linking times by half or more.


As a future improvement, gold also allows for incremental linking. 
Turning on that is outside the scope of this issue.


Bug: https://bugs.openjdk.java.net/browse/JDK-8217723
Patch inline:
diff --git a/make/autoconf/flags-ldflags.m4 
b/make/autoconf/flags-ldflags.m4

--- a/make/autoconf/flags-ldflags.m4
+++ b/make/autoconf/flags-ldflags.m4
@@ -70,7 +70,7 @@
 fi

 # Add -z defs, to forbid undefined symbols in object files.
-    BASIC_LDFLAGS="$BASIC_LDFLAGS -Wl,-z,defs"
+    BASIC_LDFLAGS="$BASIC_LDFLAGS -Wl,-z,defs -fuse-ld=gold"

 BASIC_LDFLAGS_JVM_ONLY="-Wl,-O1 -Wl,-z,relro"

/Magnus

RFR: JDK-8217723 Switch ld from bfd to gold on gcc toolchain

[Magnus Ihse Bursie]

The default binutils linker used by gcc, the bfd linker, is slow. The 
new replacement, gold, has been distributed alongside gcc for several 
years now, and is a well mature, and much faster, replacement.


This issue is about replacing ld.bfd with ld.gold for our gcc toolchain. 
In general, this cuts linking times by half or more.


As a future improvement, gold also allows for incremental linking. 
Turning on that is outside the scope of this issue.


Bug: https://bugs.openjdk.java.net/browse/JDK-8217723
Patch inline:
diff --git a/make/autoconf/flags-ldflags.m4 b/make/autoconf/flags-ldflags.m4
--- a/make/autoconf/flags-ldflags.m4
+++ b/make/autoconf/flags-ldflags.m4
@@ -70,7 +70,7 @@
 fi

 # Add -z defs, to forbid undefined symbols in object files.
-    BASIC_LDFLAGS="$BASIC_LDFLAGS -Wl,-z,defs"
+    BASIC_LDFLAGS="$BASIC_LDFLAGS -Wl,-z,defs -fuse-ld=gold"

 BASIC_LDFLAGS_JVM_ONLY="-Wl,-O1 -Wl,-z,relro"

/Magnus

Re: Fail to cross compiling jdk12 for mips64el-linux-gnu target

[Leslie Zhai]

Hi Magnus,

Thanks for your kind response!

在 2019/1/24 下午8:05, Magnus Ihse Bursie 写道:



On 2019-01-24 05:45, David Holmes wrote:

On 24/01/2019 1:51 pm, Ao Qi wrote:

Hi David,

On Thu, Jan 24, 2019 at 10:47 AM David Holmes 
 wrote:


Hi Leslie,

I'm not Erik :) however 

On 24/01/2019 12:28 pm, Leslie Zhai wrote:

Hi Erik,

Because the flags' macro of MIPS is different from other 
targets[1], it

will effect the `UnixConstants` of
src/java.base/unix/classes/sun/nio/fs/UnixConstants.java.template, 
for
example, PREFIX_O_APPEND, so the `Flags` in `FileChannel` is 
different

from other targets.  Then failed[2] to cross compiling jdk12 for
mips64el-linux-gnu target by following the document[3].


It's been quite a while since we built for a platform where the flags
had different values, but it is supposed to work. I suspect that 
the way

we build now with the module system may not be quite correct in this
regard - but that is just supposition on my part. Because compile-time
constants are stored directly into class files that use them, we 
have to
generate UnixConstants.java before compiling any class that needs 
to use
it, then we must compile all classes that do use it. Those classes 
must

only be executed in conjunction with a tool executing on the target
platform (javac, jmod etc). My suspicion is that we may be executing a
tool on the build platform using the newly compiled classes for the
target platform - and that would be wrong.



Yes. Some variables in UnixConstants.java is defined according to the
target (fcntl.h of mips in this case), but would be executed on host
(x86 in this case) during "make images". These variables of ppc64le,
aarch64 and s390x are the same as x86, but the ones of mips are not.
However, mips is not an official supported target and I did not find
other targets have similar issue, I am not sure if this is a bug or if
this is the right mailing list :)


This is the right mailing list to discuss. :)

So I went back to find when this happened in the past and I 
mis-remembered how we solved it. We previously had a problem with the 
Oracle Java SE Embedded PPC port where some Linux PPC-e500v2 
platforms defined a different value for the O_NOFOLLOW and O_DIRECT 
flags. The solution then was to commit a platform specific version of 
UnixConstants.java and change the build system to only use the 
template file if there was not a pre-existing .java file. But that 
platform and the SE Embedded support was all removed. 
(SocketOptions.java had a similar problem.) Further that build logic 
is completely different to what we had back then (JDK 6/7).


This is definitely a bug in our build logic IMHO as we specifically 
use the build platform c-pre-processor to process the template file 
based on the target header files, but then use the resulting class 
when running tools on the build platform. 
I agree with David. This is a bug in the build system; arguably even a 
regression.


This was not something that happened in 6/7 or even 8 and I think the 
module system tools are where the issue now lies.


I think Erik and/or Magnus will have to give advice on how this may 
be properly fixed. I can't see any simple solution.

Unfortunately, I also agree that there is no simple solution. :-(

The proper way to handle this is to generate two versions of 
UnixConstants and friends, one for the build platform, and one for the 
target platform. But as far as I know, there is no simple way to 
handle such complications in our current structure. Erik knows more 
about this, he was the one who designed the current solution for 
handling cross-compilation in the modularized world.


However, I can at least help you with a relatively simple workaround. 
First some background: When we cross-compile, we need not only a Boot 
JDK (of version current N-1) running on the build host platform, but 
we also need a Build JDK, based on the current source code, running on 
the build host. (This is for running jmod/jlink; it needs to be up to 
date). If we have no Build JDK present, we start by creating one 
ourselves. This is more or less the same output as running a normal, 
non-cross compiled, build on the build host. (But Erik found some ways 
to cut a few corners to save time.) Your problem is that the Build JDK 
is broken, since UnixConstants has the wrong values.

-
But you can supply your own working Build JDK to configure!

So, your workflow for cross-compiling to MIPS should be:
1) create a Build JDK, e.g. like this: "bash configure 
--with-conf-name=buildjdk && make jdk CONF=buildjdk"
2) create your cross-compilation JDK like this: "bash configure 
--openjdk-target=mipsel-unknown-linux-gnu 
--with-build-jdk=build/buildjdk/jdk"

... or something like that.
Then you can build your cross-compiled jdk with "make images CONF=mips".


I will try that, thanks a lot for your teaching!

Regards,

Leslie Zhai





In theory, you should rebuild your buildjdk for each and every change 
("make jdk 

Re: Fail to cross compiling jdk12 for mips64el-linux-gnu target

[Magnus Ihse Bursie]

On 2019-01-24 05:45, David Holmes wrote:

On 24/01/2019 1:51 pm, Ao Qi wrote:

Hi David,

On Thu, Jan 24, 2019 at 10:47 AM David Holmes 
 wrote:


Hi Leslie,

I'm not Erik :) however 

On 24/01/2019 12:28 pm, Leslie Zhai wrote:

Hi Erik,

Because the flags' macro of MIPS is different from other 
targets[1], it

will effect the `UnixConstants` of
src/java.base/unix/classes/sun/nio/fs/UnixConstants.java.template, for
example, PREFIX_O_APPEND, so the `Flags` in `FileChannel` is different
from other targets.  Then failed[2] to cross compiling jdk12 for
mips64el-linux-gnu target by following the document[3].


It's been quite a while since we built for a platform where the flags
had different values, but it is supposed to work. I suspect that the 
way

we build now with the module system may not be quite correct in this
regard - but that is just supposition on my part. Because compile-time
constants are stored directly into class files that use them, we 
have to
generate UnixConstants.java before compiling any class that needs to 
use

it, then we must compile all classes that do use it. Those classes must
only be executed in conjunction with a tool executing on the target
platform (javac, jmod etc). My suspicion is that we may be executing a
tool on the build platform using the newly compiled classes for the
target platform - and that would be wrong.



Yes. Some variables in UnixConstants.java is defined according to the
target (fcntl.h of mips in this case), but would be executed on host
(x86 in this case) during "make images". These variables of ppc64le,
aarch64 and s390x are the same as x86, but the ones of mips are not.
However, mips is not an official supported target and I did not find
other targets have similar issue, I am not sure if this is a bug or if
this is the right mailing list :)


This is the right mailing list to discuss. :)

So I went back to find when this happened in the past and I 
mis-remembered how we solved it. We previously had a problem with the 
Oracle Java SE Embedded PPC port where some Linux PPC-e500v2 platforms 
defined a different value for the O_NOFOLLOW and O_DIRECT flags. The 
solution then was to commit a platform specific version of 
UnixConstants.java and change the build system to only use the 
template file if there was not a pre-existing .java file. But that 
platform and the SE Embedded support was all removed. 
(SocketOptions.java had a similar problem.) Further that build logic 
is completely different to what we had back then (JDK 6/7).


This is definitely a bug in our build logic IMHO as we specifically 
use the build platform c-pre-processor to process the template file 
based on the target header files, but then use the resulting class 
when running tools on the build platform. 
I agree with David. This is a bug in the build system; arguably even a 
regression.


This was not something that happened in 6/7 or even 8 and I think the 
module system tools are where the issue now lies.


I think Erik and/or Magnus will have to give advice on how this may be 
properly fixed. I can't see any simple solution.

Unfortunately, I also agree that there is no simple solution. :-(

The proper way to handle this is to generate two versions of 
UnixConstants and friends, one for the build platform, and one for the 
target platform. But as far as I know, there is no simple way to handle 
such complications in our current structure. Erik knows more about this, 
he was the one who designed the current solution for handling 
cross-compilation in the modularized world.


However, I can at least help you with a relatively simple workaround. 
First some background: When we cross-compile, we need not only a Boot 
JDK (of version current N-1) running on the build host platform, but we 
also need a Build JDK, based on the current source code, running on the 
build host. (This is for running jmod/jlink; it needs to be up to date). 
If we have no Build JDK present, we start by creating one ourselves. 
This is more or less the same output as running a normal, non-cross 
compiled, build on the build host. (But Erik found some ways to cut a 
few corners to save time.) Your problem is that the Build JDK is broken, 
since UnixConstants has the wrong values.

-
But you can supply your own working Build JDK to configure!

So, your workflow for cross-compiling to MIPS should be:
1) create a Build JDK, e.g. like this: "bash configure 
--with-conf-name=buildjdk && make jdk CONF=buildjdk"
2) create your cross-compilation JDK like this: "bash configure 
--openjdk-target=mipsel-unknown-linux-gnu 
--with-build-jdk=build/buildjdk/jdk"

... or something like that.
Then you can build your cross-compiled jdk with "make images CONF=mips".

In theory, you should rebuild your buildjdk for each and every change 
("make jdk CONF=buildjdk && make images CONF=mips"); but in reality, I 
don't believe there is really any need to rebuild the buildjdk (unless 
major changes in jlink/jmod happens).


/Magnus