On 2020-06-01 15:36, Yasumasa Suenaga wrote:
Hi Magnus,
On 2020/06/01 20:06, Magnus Ihse Bursie wrote:
On 2020-05-31 14:39, Yasumasa Suenaga wrote:
Hi all,
I tried to build jdk/jdk on WSL 2 with VS2019, but I saw three
errors as below.
1) Could not detect VC (cl.exe) version. So configure reports that
it could not find valid Microsoft C++ compiler.
2) Could not generate fixpath.exe because
$CONFIGURESUPPORT_OUTPUTDIR/bin did not exist. `wslpath -m` in
UTIL_REWRITE_AS_WINDOWS_MIXED_PATH - it does not seem to work with
unavailable path.
3) LPVOID value is passed to FormatMessage() in fixpath.c .
FormatMessage() accepts value for lpBuffer as LPTSTR.
I think it is worth to fix. Someone already has worked for this?
I have been experimenting with building on WSL2. I've noticed the
same issues as you have, but solved them somewhat differently. For
instance, you cannot grep for "Optimizing Compiler" since this phrase
is localized. We've had, and fixed, that bug before.
I run on cl.exe on Japanese locale, but it is not localized.
(Is it because Japanese...? but the usage is localized.)
```
$
/mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe
Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28805 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
使い方: cl [ オプション... ] ファイル名... [ /link リンク オプション... ]
```
I can't say anything about the Japanese locale, but I know for a fact
that we've had localization issues. I think it was the French or
Portuguese locale that had a "Compileur Optimisée" or something like that.
Incidentally, that fix is actually a workaround for what looks like a
worrying issue in WSL2: race conditions on piping. Check this output:
magnusi@winihse:/mnt/c/localdata/hg/sandbox-ALT$
/mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe
2>&1 | cat
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
usage: cl [ option... ] filename... [ /link linkoption... ]
Copyright (C) Microsoft Corporation. All rights reserved.
magnusi@winihse:/mnt/c/localdata/hg/sandbox-ALT$
/mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe
2>&1 | cat
usage: cl [ option... ] filename... [ /link linkoption... ]
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
magnusi@winihse:/mnt/c/localdata/hg/sandbox-ALT$
/mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
usage: cl [ option... ] filename... [ /link linkoption... ]
magnusi@winihse:/mnt/c/localdata/hg/sandbox-ALT$
/mnt/c/jib/jib-ma~1/install/jpg/infra/buildd~1/devkit~2/vs2019~1.0/devkit~1.gz/vc/bin/x64/cl.exe
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28612 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
usage: cl [ option... ] filename... [ /link linkoption... ]
Notice how the order of lines are random when piping through cat.
Worrying, indeed. I wonder what more will break...
AFAICS the usage is in stdout, the banner is in stderr.
```
$
/mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe
2>&1 1>/dev/null
Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28805 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
$
/mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe
2>/dev/null
使い方: cl [ オプション... ] ファイル名... [ /link リンク オプション... ]
```
So we can catch up version string from cl.exe as below:
```
$
/mnt/c/progra~2/micros~2/2019/commun~1/vc/tools/msvc/1426~1.288/bin/hostx86/x64/cl.exe
2>&1 1>/dev/null | head -n 1
Microsoft(R) C/C++ Optimizing Compiler Version 19.26.28805 for x64
```
Good thinking. That is probably more stable than grepping for "microsoft".
Also, wslpath suddenyl not working with non-existing paths looks like
a pure bug to me. :( But the correct workaround is not to go around
and touch arbitrary files in the file system. For one thing, you
might not have write permission there. I think the proper way (short
of awaiting a bug fix from Microsoft) is to iteratively remove the
last part of the path until wslpath succeeds, and then add back on
what we removed.
Or we need to check how we are using wslpath. If it is used to verify
if a file is present, we can use the failure from wslpath as a sign
that the file is missing, instead of trying first to convert it to
unix path, and then check if it is present.
In my understand, wslpath for FIXPATH_BIN_W is to convert to the path
for Windows (for cl.exe).
So I think it is nature even if it does not exist.
In addition, if we do not have write permission, it should be failed.
We need to have support for general paths, regardless if they exist or
if we have write permission. It might be to convert a given path to a
configure option or whatever. So we need to be able to handle these
cases too.
---
However, the main problem when working with WSL2 is that it is dead
slow, for building OpenJDK. :-( The method that was used in WSL1, to
have the source code and build directory on a Windows share, is not
working at all in WSL2. Access of NTFS drives has horrendous
performance. :( On the flip side, WSL2 adds access to the Linux drive
for Windows apps. So it might be possible to do the reverse -- make
sure we store source and build directory on a Linux drive, and let
the windows apps access it.
I have started testing this but have not had time to finish my
testing. A lot of the Windows integration code in autoconf do not
work when you do this.
Yeah, I haven't yet succeeded to build OpenJDK on WSL2 because I/O is
very slow... but I believe Microsoft will be improve I/O performance
some day :)
Your suggestion to access to WSL partition from Windows sounds good to
me, but I think configure script should be detect it runs on whether
WSL 1 or 2.
However I couldn't find how can we detect it smartly.
Yes, configure should check what version of WSL we are running on. And
yes, it's not trivial to detect. Idiotic move by Microsoft to not make
this information accessible. :-( But I've found ways to do this.
I'm implementing this, and other fixes, in the jdk/sandbox repo, in the
branch ihse-wsl-branch. You are free to check it out and test how it
works for you. (In fact, your assistance would be appreciated!) Please
note that this is a work in progress, and it can/will break from time to
time. :-) I make changes in the branch and then try to verify them in
different environments.
/Magnus
Thanks,
Yasumasa
/Magnus
If not, I will file it to JBS, and will send review request.
Thanks,
Yasumasa
