Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Neat! Thank you. ..Thomas On Wed, Jan 16, 2019 at 5:36 PM Erik Joelsson wrote: > Build support is in and you should be able to follow the documentation in > doc/building.md. Test support is still waiting on a jtreg release and a > patch to use it, but even with that, there are tests (at least shell tests) > that will not work properly. > > /Erik > On 2019-01-16 00:49, Thomas Stüfe wrote: > > Hi guys, > > Just wanted to know what the state is. Did you already add the support for > WSL or is this still WIP? If it should work, is there a documentation > somewhere I could follow? > > Thank you! > > Thomas > > > On Sat, Dec 22, 2018 at 4:55 AM Andrew Luo < > andrewluotechnolog...@outlook.com> wrote: > >> Just wanted to update the thread with the issues we discovered with WSL >> while adding support to the OpenJDK build system. I reported these issues >> to the WSL team for all except for one of the bugs, which I'm still >> investigating. >> >> GenerateCurrencyData.java - issue with Properties.load(System.in): >> https://github.com/Microsoft/WSL/issues/3723 >> Issue with directly calling cmd.exe to transform long path to short path: >> https://github.com/Microsoft/WSL/issues/3724 >> Calling from Linux to Win32 with untransformable Linux paths in a WSLENV >> path environment variable causes extra output: >> https://github.com/Microsoft/WSL/issues/3725 >> Spp.java - still investigating this >> >> Thanks, >> >> -Andrew >> >> -Original Message- >> From: Erik Joelsson >> Sent: Thursday, December 20, 2018 1:51 AM >> To: Andrew Luo ; Magnus Ihse Bursie < >> magnus.ihse.bur...@oracle.com> >> Cc: build-dev@openjdk.java.net >> Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem >> for Linux) on Windows >> >> I've updated and two builds in a row have now succeeded. I will keep >> running, but it does seem likely that the new version has fixed the issue. >> >> /Erik >> >> On 2018-12-20 09:44, Erik Joelsson wrote: >> > Hello, >> > >> > On 2018-12-19 19:40, Andrew Luo wrote: >> >> Hi Erik, >> >> >> >> Which target are you using (make exploded-image?)? I never saw this >> >> error while building on my machine (I've built about 10 times now, >> >> I'm on Windows 10 1809 for what it's worth). Perhaps I can try to >> >> reproduce this on my system as well... >> > >> > The target doesn't really matter that much, it's failing when building >> > java modules, so exploded-image should reproduce it. I have built >> > successfully as well, so this only happens intermittently. Here is the >> > environment string from my system: >> > >> > WSL version Ubuntu 16.04.4 LTS #471-Microsoft Fri Dec 07 20:04:00 PST >> > 2018 4.4.0-17134-Microsoft (on Windows build 10.0.17134.471) >> > >> > In System about, it identifies itself as Windows 10 Pro 1803, so that >> > looks older than yours. I will see if I can update. >> > >> > I should also note that deleting build output is not necessary (and >> > probably not affecting) success or failure on rebuild. From what I can >> > see what happens is: make runs the find command to find all java >> > source files and puts that list of files as prerequisites to the java >> > compile rule, then when evaluating the rule, it sometimes fails to >> > resolve a file. This would seem like a bug in the filesystem to me. >> > >> > /Erik >> > >> >> Thanks, >> >> >> >> -Andrew >> >> >> >> -Original Message- >> >> From: Erik Joelsson >> >> Sent: Wednesday, December 19, 2018 8:28 AM >> >> To: Andrew Luo ; Magnus Ihse >> >> Bursie >> >> Cc: build-dev@openjdk.java.net >> >> Subject: Re: [PATCH] Support for building using WSL (Windows >> >> Subsystem for Linux) on Windows >> >> >> >> I'm now seeing intermittent build failures that look like this: >> >> >> >> make[3]: *** No rule to make target >> >> '/mnt/d/erik/jdk-sandbox/open/src/java.security.jgss/share/classes/su >> >> n/security/krb5/internal/TGSReq.java', >> >> >> >> needed by >> >> >> '/mnt/d/erik/jdk-sandbox/build/windows-x86_64-server-release/jdk/modules/java.security.jgss/_the.java.security.jgss_batch'. >> >> >> >> >> Stop. >> >> >> >> The part
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Erik, I can push/promote what we have so far for jtreg, but it would be good to decide what (if anything) we are going to do with the shell tests. -- Jon On 1/16/19 8:41 AM, Erik Joelsson wrote: Build support is in and you should be able to follow the documentation in doc/building.md. Test support is still waiting on a jtreg release and a patch to use it, but even with that, there are tests (at least shell tests) that will not work properly. /Erik On 2019-01-16 00:49, Thomas Stüfe wrote: Hi guys, Just wanted to know what the state is. Did you already add the support for WSL or is this still WIP? If it should work, is there a documentation somewhere I could follow? Thank you! Thomas On Sat, Dec 22, 2018 at 4:55 AM Andrew Luo <mailto:andrewluotechnolog...@outlook.com>> wrote: Just wanted to update the thread with the issues we discovered with WSL while adding support to the OpenJDK build system. I reported these issues to the WSL team for all except for one of the bugs, which I'm still investigating. GenerateCurrencyData.java - issue with Properties.load(System.in): https://github.com/Microsoft/WSL/issues/3723 Issue with directly calling cmd.exe to transform long path to short path: https://github.com/Microsoft/WSL/issues/3724 Calling from Linux to Win32 with untransformable Linux paths in a WSLENV path environment variable causes extra output: https://github.com/Microsoft/WSL/issues/3725 Spp.java - still investigating this Thanks, -Andrew -Original Message- From: Erik Joelsson mailto:erik.joels...@oracle.com>> Sent: Thursday, December 20, 2018 1:51 AM To: Andrew Luo mailto:andrewluotechnolog...@outlook.com>>; Magnus Ihse Bursie <mailto:magnus.ihse.bur...@oracle.com>> Cc: build-dev@openjdk.java.net <mailto:build-dev@openjdk.java.net> Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I've updated and two builds in a row have now succeeded. I will keep running, but it does seem likely that the new version has fixed the issue. /Erik On 2018-12-20 09:44, Erik Joelsson wrote: > Hello, > > On 2018-12-19 19:40, Andrew Luo wrote: >> Hi Erik, >> >> Which target are you using (make exploded-image?)? I never saw this >> error while building on my machine (I've built about 10 times now, >> I'm on Windows 10 1809 for what it's worth). Perhaps I can try to >> reproduce this on my system as well... > > The target doesn't really matter that much, it's failing when building > java modules, so exploded-image should reproduce it. I have built > successfully as well, so this only happens intermittently. Here is the > environment string from my system: > > WSL version Ubuntu 16.04.4 LTS #471-Microsoft Fri Dec 07 20:04:00 PST > 2018 4.4.0-17134-Microsoft (on Windows build 10.0.17134.471) > > In System about, it identifies itself as Windows 10 Pro 1803, so that > looks older than yours. I will see if I can update. > > I should also note that deleting build output is not necessary (and > probably not affecting) success or failure on rebuild. From what I can > see what happens is: make runs the find command to find all java > source files and puts that list of files as prerequisites to the java > compile rule, then when evaluating the rule, it sometimes fails to > resolve a file. This would seem like a bug in the filesystem to me. > > /Erik > >> Thanks, >> >> -Andrew >> >> -Original Message- >> From: Erik Joelsson mailto:erik.joels...@oracle.com>> >> Sent: Wednesday, December 19, 2018 8:28 AM >> To: Andrew Luo mailto:andrewluotechnolog...@outlook.com>>; Magnus Ihse >> Bursie mailto:magnus.ihse.bur...@oracle.com>> >> Cc: build-dev@openjdk.java.net <mailto:build-dev@openjdk.java.net> >> Subject: Re: [PATCH] Support for building using WSL (Windows >> Subsystem for Linux) on Windows >> >> I'm now seeing intermittent build failures that look like this: >> >> make[3]: *** No rule to make target >> '/mnt/d/erik/jdk-sandbox/open/src/java.security.jgss/share/classes/su >> n/security/krb5/internal/TGSReq.java', >> >> needed by >> '/mnt/d/erik/jdk-sandbox/build/windows-x86_64-server-release/jdk/modules/java.security.jgss/_the.java.security.jgss_batch'. >> >> Stop. >> >> The particular file that's missing varies, and deleting the output >> dir for that module and re
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Build support is in and you should be able to follow the documentation
in doc/building.md. Test support is still waiting on a jtreg release and
a patch to use it, but even with that, there are tests (at least shell
tests) that will not work properly.
/Erik
On 2019-01-16 00:49, Thomas Stüfe wrote:
Hi guys,
Just wanted to know what the state is. Did you already add the support
for WSL or is this still WIP? If it should work, is there a
documentation somewhere I could follow?
Thank you!
Thomas
On Sat, Dec 22, 2018 at 4:55 AM Andrew Luo
<mailto:andrewluotechnolog...@outlook.com>> wrote:
Just wanted to update the thread with the issues we discovered
with WSL while adding support to the OpenJDK build system. I
reported these issues to the WSL team for all except for one of
the bugs, which I'm still investigating.
GenerateCurrencyData.java - issue with Properties.load(System.in):
https://github.com/Microsoft/WSL/issues/3723
Issue with directly calling cmd.exe to transform long path to
short path: https://github.com/Microsoft/WSL/issues/3724
Calling from Linux to Win32 with untransformable Linux paths in a
WSLENV path environment variable causes extra output:
https://github.com/Microsoft/WSL/issues/3725
Spp.java - still investigating this
Thanks,
-Andrew
-Original Message-
From: Erik Joelsson mailto:erik.joels...@oracle.com>>
Sent: Thursday, December 20, 2018 1:51 AM
To: Andrew Luo mailto:andrewluotechnolog...@outlook.com>>; Magnus Ihse Bursie
mailto:magnus.ihse.bur...@oracle.com>>
Cc: build-dev@openjdk.java.net <mailto:build-dev@openjdk.java.net>
Subject: Re: [PATCH] Support for building using WSL (Windows
Subsystem for Linux) on Windows
I've updated and two builds in a row have now succeeded. I will
keep running, but it does seem likely that the new version has
fixed the issue.
/Erik
On 2018-12-20 09:44, Erik Joelsson wrote:
> Hello,
>
> On 2018-12-19 19:40, Andrew Luo wrote:
>> Hi Erik,
>>
>> Which target are you using (make exploded-image?)? I never saw
this
>> error while building on my machine (I've built about 10 times now,
>> I'm on Windows 10 1809 for what it's worth). Perhaps I can try to
>> reproduce this on my system as well...
>
> The target doesn't really matter that much, it's failing when
building
> java modules, so exploded-image should reproduce it. I have built
> successfully as well, so this only happens intermittently. Here
is the
> environment string from my system:
>
> WSL version Ubuntu 16.04.4 LTS #471-Microsoft Fri Dec 07
20:04:00 PST
> 2018 4.4.0-17134-Microsoft (on Windows build 10.0.17134.471)
>
> In System about, it identifies itself as Windows 10 Pro 1803, so
that
> looks older than yours. I will see if I can update.
>
> I should also note that deleting build output is not necessary (and
> probably not affecting) success or failure on rebuild. From what
I can
> see what happens is: make runs the find command to find all java
> source files and puts that list of files as prerequisites to the
java
> compile rule, then when evaluating the rule, it sometimes fails to
> resolve a file. This would seem like a bug in the filesystem to me.
>
> /Erik
>
>> Thanks,
>>
>> -Andrew
>>
>> -Original Message-
>> From: Erik Joelsson mailto:erik.joels...@oracle.com>>
>> Sent: Wednesday, December 19, 2018 8:28 AM
>> To: Andrew Luo mailto:andrewluotechnolog...@outlook.com>>; Magnus Ihse
>> Bursie mailto:magnus.ihse.bur...@oracle.com>>
>> Cc: build-dev@openjdk.java.net <mailto:build-dev@openjdk.java.net>
>> Subject: Re: [PATCH] Support for building using WSL (Windows
>> Subsystem for Linux) on Windows
>>
>> I'm now seeing intermittent build failures that look like this:
>>
>> make[3]: *** No rule to make target
>>
'/mnt/d/erik/jdk-sandbox/open/src/java.security.jgss/share/classes/su
>> n/security/krb5/internal/TGSReq.java',
>>
>> needed by
>>
'/mnt/d/erik/jdk-sandbox/build/windows-x86_64-server-release/jdk/modules/java.security.jgss/_the.java.security.jgss_batch'.
>>
>> Stop.
>>
>> The particular file that's missing varies, and deleting the output
>> dir for that module and rebuild works. The common pattern seems
to be
>> upper case letters in the file name of the source file.
>>
>> I will investigate some more.
>>
>> /Erik
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hi guys, Just wanted to know what the state is. Did you already add the support for WSL or is this still WIP? If it should work, is there a documentation somewhere I could follow? Thank you! Thomas On Sat, Dec 22, 2018 at 4:55 AM Andrew Luo < andrewluotechnolog...@outlook.com> wrote: > Just wanted to update the thread with the issues we discovered with WSL > while adding support to the OpenJDK build system. I reported these issues > to the WSL team for all except for one of the bugs, which I'm still > investigating. > > GenerateCurrencyData.java - issue with Properties.load(System.in): > https://github.com/Microsoft/WSL/issues/3723 > Issue with directly calling cmd.exe to transform long path to short path: > https://github.com/Microsoft/WSL/issues/3724 > Calling from Linux to Win32 with untransformable Linux paths in a WSLENV > path environment variable causes extra output: > https://github.com/Microsoft/WSL/issues/3725 > Spp.java - still investigating this > > Thanks, > > -Andrew > > -Original Message- > From: Erik Joelsson > Sent: Thursday, December 20, 2018 1:51 AM > To: Andrew Luo ; Magnus Ihse Bursie < > magnus.ihse.bur...@oracle.com> > Cc: build-dev@openjdk.java.net > Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for > Linux) on Windows > > I've updated and two builds in a row have now succeeded. I will keep > running, but it does seem likely that the new version has fixed the issue. > > /Erik > > On 2018-12-20 09:44, Erik Joelsson wrote: > > Hello, > > > > On 2018-12-19 19:40, Andrew Luo wrote: > >> Hi Erik, > >> > >> Which target are you using (make exploded-image?)? I never saw this > >> error while building on my machine (I've built about 10 times now, > >> I'm on Windows 10 1809 for what it's worth). Perhaps I can try to > >> reproduce this on my system as well... > > > > The target doesn't really matter that much, it's failing when building > > java modules, so exploded-image should reproduce it. I have built > > successfully as well, so this only happens intermittently. Here is the > > environment string from my system: > > > > WSL version Ubuntu 16.04.4 LTS #471-Microsoft Fri Dec 07 20:04:00 PST > > 2018 4.4.0-17134-Microsoft (on Windows build 10.0.17134.471) > > > > In System about, it identifies itself as Windows 10 Pro 1803, so that > > looks older than yours. I will see if I can update. > > > > I should also note that deleting build output is not necessary (and > > probably not affecting) success or failure on rebuild. From what I can > > see what happens is: make runs the find command to find all java > > source files and puts that list of files as prerequisites to the java > > compile rule, then when evaluating the rule, it sometimes fails to > > resolve a file. This would seem like a bug in the filesystem to me. > > > > /Erik > > > >> Thanks, > >> > >> -Andrew > >> > >> -Original Message- > >> From: Erik Joelsson > >> Sent: Wednesday, December 19, 2018 8:28 AM > >> To: Andrew Luo ; Magnus Ihse > >> Bursie > >> Cc: build-dev@openjdk.java.net > >> Subject: Re: [PATCH] Support for building using WSL (Windows > >> Subsystem for Linux) on Windows > >> > >> I'm now seeing intermittent build failures that look like this: > >> > >> make[3]: *** No rule to make target > >> '/mnt/d/erik/jdk-sandbox/open/src/java.security.jgss/share/classes/su > >> n/security/krb5/internal/TGSReq.java', > >> > >> needed by > >> > '/mnt/d/erik/jdk-sandbox/build/windows-x86_64-server-release/jdk/modules/java.security.jgss/_the.java.security.jgss_batch'. > > >> > >> Stop. > >> > >> The particular file that's missing varies, and deleting the output > >> dir for that module and rebuild works. The common pattern seems to be > >> upper case letters in the file name of the source file. > >> > >> I will investigate some more. > >> > >> /Erik > >> > >> On 2018-12-19 06:18, Erik Joelsson wrote: > >>> I can also report that on the Windows 10 machine I'm testing this > >>> on, "make bundles" on Cygwin took 12m56s and in WSL 8m39s, so a > >>> pretty big improvement! > >>> > >>> /Erik > >>> > >>> On 2018-12-19 03:45, Erik Joelsson wrote: > >>>> Hello, > >>>> > >>>> On 2018-12-19 00:19, Erik Joelsson wrote: > >>>>
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Just wanted to update the thread with the issues we discovered with WSL while adding support to the OpenJDK build system. I reported these issues to the WSL team for all except for one of the bugs, which I'm still investigating. GenerateCurrencyData.java - issue with Properties.load(System.in): https://github.com/Microsoft/WSL/issues/3723 Issue with directly calling cmd.exe to transform long path to short path: https://github.com/Microsoft/WSL/issues/3724 Calling from Linux to Win32 with untransformable Linux paths in a WSLENV path environment variable causes extra output: https://github.com/Microsoft/WSL/issues/3725 Spp.java - still investigating this Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Thursday, December 20, 2018 1:51 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I've updated and two builds in a row have now succeeded. I will keep running, but it does seem likely that the new version has fixed the issue. /Erik On 2018-12-20 09:44, Erik Joelsson wrote: > Hello, > > On 2018-12-19 19:40, Andrew Luo wrote: >> Hi Erik, >> >> Which target are you using (make exploded-image?)? I never saw this >> error while building on my machine (I've built about 10 times now, >> I'm on Windows 10 1809 for what it's worth). Perhaps I can try to >> reproduce this on my system as well... > > The target doesn't really matter that much, it's failing when building > java modules, so exploded-image should reproduce it. I have built > successfully as well, so this only happens intermittently. Here is the > environment string from my system: > > WSL version Ubuntu 16.04.4 LTS #471-Microsoft Fri Dec 07 20:04:00 PST > 2018 4.4.0-17134-Microsoft (on Windows build 10.0.17134.471) > > In System about, it identifies itself as Windows 10 Pro 1803, so that > looks older than yours. I will see if I can update. > > I should also note that deleting build output is not necessary (and > probably not affecting) success or failure on rebuild. From what I can > see what happens is: make runs the find command to find all java > source files and puts that list of files as prerequisites to the java > compile rule, then when evaluating the rule, it sometimes fails to > resolve a file. This would seem like a bug in the filesystem to me. > > /Erik > >> Thanks, >> >> -Andrew >> >> -Original Message- >> From: Erik Joelsson >> Sent: Wednesday, December 19, 2018 8:28 AM >> To: Andrew Luo ; Magnus Ihse >> Bursie >> Cc: build-dev@openjdk.java.net >> Subject: Re: [PATCH] Support for building using WSL (Windows >> Subsystem for Linux) on Windows >> >> I'm now seeing intermittent build failures that look like this: >> >> make[3]: *** No rule to make target >> '/mnt/d/erik/jdk-sandbox/open/src/java.security.jgss/share/classes/su >> n/security/krb5/internal/TGSReq.java', >> >> needed by >> '/mnt/d/erik/jdk-sandbox/build/windows-x86_64-server-release/jdk/modules/java.security.jgss/_the.java.security.jgss_batch'. >> >> >> Stop. >> >> The particular file that's missing varies, and deleting the output >> dir for that module and rebuild works. The common pattern seems to be >> upper case letters in the file name of the source file. >> >> I will investigate some more. >> >> /Erik >> >> On 2018-12-19 06:18, Erik Joelsson wrote: >>> I can also report that on the Windows 10 machine I'm testing this >>> on, "make bundles" on Cygwin took 12m56s and in WSL 8m39s, so a >>> pretty big improvement! >>> >>> /Erik >>> >>> On 2018-12-19 03:45, Erik Joelsson wrote: >>>> Hello, >>>> >>>> On 2018-12-19 00:19, Erik Joelsson wrote: >>>>> Hello Andrew, >>>>> >>>>> On 2018-12-18 12:45, Andrew Luo wrote: >>>>>> Hi Erik/Magnus, >>>>>> >>>>>> I've attached my latest changes: >>>>>> >>>>>> 1. Fixed a file I forgot to revert in my previous change while >>>>>> trying something out... >>>>>> 2. Added information about case sensitivity of the OpenJDK build >>>>>> directory (yes, I did use the make target to generate the HTML >>>>>> file) 3. Fixed Cygwin (hopefully, I don't have a Cygwin >>>>>> environment to verify this) >>>>> I will take this patch for a spin and see. >>>>> >>>> After applying a fix for >>>>
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
I've updated and two builds in a row have now succeeded. I will keep running, but it does seem likely that the new version has fixed the issue. /Erik On 2018-12-20 09:44, Erik Joelsson wrote: Hello, On 2018-12-19 19:40, Andrew Luo wrote: Hi Erik, Which target are you using (make exploded-image?)? I never saw this error while building on my machine (I've built about 10 times now, I'm on Windows 10 1809 for what it's worth). Perhaps I can try to reproduce this on my system as well... The target doesn't really matter that much, it's failing when building java modules, so exploded-image should reproduce it. I have built successfully as well, so this only happens intermittently. Here is the environment string from my system: WSL version Ubuntu 16.04.4 LTS #471-Microsoft Fri Dec 07 20:04:00 PST 2018 4.4.0-17134-Microsoft (on Windows build 10.0.17134.471) In System about, it identifies itself as Windows 10 Pro 1803, so that looks older than yours. I will see if I can update. I should also note that deleting build output is not necessary (and probably not affecting) success or failure on rebuild. From what I can see what happens is: make runs the find command to find all java source files and puts that list of files as prerequisites to the java compile rule, then when evaluating the rule, it sometimes fails to resolve a file. This would seem like a bug in the filesystem to me. /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Wednesday, December 19, 2018 8:28 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I'm now seeing intermittent build failures that look like this: make[3]: *** No rule to make target '/mnt/d/erik/jdk-sandbox/open/src/java.security.jgss/share/classes/sun/security/krb5/internal/TGSReq.java', needed by '/mnt/d/erik/jdk-sandbox/build/windows-x86_64-server-release/jdk/modules/java.security.jgss/_the.java.security.jgss_batch'. Stop. The particular file that's missing varies, and deleting the output dir for that module and rebuild works. The common pattern seems to be upper case letters in the file name of the source file. I will investigate some more. /Erik On 2018-12-19 06:18, Erik Joelsson wrote: I can also report that on the Windows 10 machine I'm testing this on, "make bundles" on Cygwin took 12m56s and in WSL 8m39s, so a pretty big improvement! /Erik On 2018-12-19 03:45, Erik Joelsson wrote: Hello, On 2018-12-19 00:19, Erik Joelsson wrote: Hello Andrew, On 2018-12-18 12:45, Andrew Luo wrote: Hi Erik/Magnus, I've attached my latest changes: 1. Fixed a file I forgot to revert in my previous change while trying something out... 2. Added information about case sensitivity of the OpenJDK build directory (yes, I did use the make target to generate the HTML file) 3. Fixed Cygwin (hopefully, I don't have a Cygwin environment to verify this) I will take this patch for a spin and see. After applying a fix for https://bugs.openjdk.java.net/browse/JDK-8215635, I managed to build everything as well. I pushed some minor adjustments to make Cygwin work too. I will need to take this through more thorough testing. /Erik With this patch I've tested the following targets: exploded-image (default): Works images: Works bundles: Works test: Completes, but some tests fail. I didn't go through the test failures completely, but all of the ones I did see are due to attempting to call CreateProcess with "sh" as the argument (in jtreg: http://hg.openjdk.java.net/code-tools/jtreg/file/36c592d2f544/src/share/classes/com/sun/javatest/regtest/exec/ShellAction.java). This isn't supported in Windows, perhaps using a Linux boot JDK would fix this (but might break other things). I can look into fixing it (on WSL you can call "wsl sh", for example), but I think since it's a completely separate repo anyways, it would be best to take up those changes separately. Let me know your thoughts on this. Ah, if a Java process is launched from a Cygwin environment, it will have the unix/cygwin tools in the path so those can be launched directly. When running in WSL, it will launch the Windows binary java.exe in the Windows environment so there is no trace of WSL. I agree that we can look into this later, but we need to note that running tests is not completely supported in WSL. /Erik Otherwise, let me know if there is any other comments/suggestions before we can merge this into the main repository. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 17, 2018 9:52 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello Andrew, On 2018-12-16 00:01, Andrew Luo wrote: For me, /mnt/c was already mounted case insensitive. Maybe this is only
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hello, On 2018-12-19 19:40, Andrew Luo wrote: Hi Erik, Which target are you using (make exploded-image?)? I never saw this error while building on my machine (I've built about 10 times now, I'm on Windows 10 1809 for what it's worth). Perhaps I can try to reproduce this on my system as well... The target doesn't really matter that much, it's failing when building java modules, so exploded-image should reproduce it. I have built successfully as well, so this only happens intermittently. Here is the environment string from my system: WSL version Ubuntu 16.04.4 LTS #471-Microsoft Fri Dec 07 20:04:00 PST 2018 4.4.0-17134-Microsoft (on Windows build 10.0.17134.471) In System about, it identifies itself as Windows 10 Pro 1803, so that looks older than yours. I will see if I can update. I should also note that deleting build output is not necessary (and probably not affecting) success or failure on rebuild. From what I can see what happens is: make runs the find command to find all java source files and puts that list of files as prerequisites to the java compile rule, then when evaluating the rule, it sometimes fails to resolve a file. This would seem like a bug in the filesystem to me. /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Wednesday, December 19, 2018 8:28 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I'm now seeing intermittent build failures that look like this: make[3]: *** No rule to make target '/mnt/d/erik/jdk-sandbox/open/src/java.security.jgss/share/classes/sun/security/krb5/internal/TGSReq.java', needed by '/mnt/d/erik/jdk-sandbox/build/windows-x86_64-server-release/jdk/modules/java.security.jgss/_the.java.security.jgss_batch'. Stop. The particular file that's missing varies, and deleting the output dir for that module and rebuild works. The common pattern seems to be upper case letters in the file name of the source file. I will investigate some more. /Erik On 2018-12-19 06:18, Erik Joelsson wrote: I can also report that on the Windows 10 machine I'm testing this on, "make bundles" on Cygwin took 12m56s and in WSL 8m39s, so a pretty big improvement! /Erik On 2018-12-19 03:45, Erik Joelsson wrote: Hello, On 2018-12-19 00:19, Erik Joelsson wrote: Hello Andrew, On 2018-12-18 12:45, Andrew Luo wrote: Hi Erik/Magnus, I've attached my latest changes: 1. Fixed a file I forgot to revert in my previous change while trying something out... 2. Added information about case sensitivity of the OpenJDK build directory (yes, I did use the make target to generate the HTML file) 3. Fixed Cygwin (hopefully, I don't have a Cygwin environment to verify this) I will take this patch for a spin and see. After applying a fix for https://bugs.openjdk.java.net/browse/JDK-8215635, I managed to build everything as well. I pushed some minor adjustments to make Cygwin work too. I will need to take this through more thorough testing. /Erik With this patch I've tested the following targets: exploded-image (default): Works images: Works bundles: Works test: Completes, but some tests fail. I didn't go through the test failures completely, but all of the ones I did see are due to attempting to call CreateProcess with "sh" as the argument (in jtreg: http://hg.openjdk.java.net/code-tools/jtreg/file/36c592d2f544/src/share/classes/com/sun/javatest/regtest/exec/ShellAction.java). This isn't supported in Windows, perhaps using a Linux boot JDK would fix this (but might break other things). I can look into fixing it (on WSL you can call "wsl sh", for example), but I think since it's a completely separate repo anyways, it would be best to take up those changes separately. Let me know your thoughts on this. Ah, if a Java process is launched from a Cygwin environment, it will have the unix/cygwin tools in the path so those can be launched directly. When running in WSL, it will launch the Windows binary java.exe in the Windows environment so there is no trace of WSL. I agree that we can look into this later, but we need to note that running tests is not completely supported in WSL. /Erik Otherwise, let me know if there is any other comments/suggestions before we can merge this into the main repository. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 17, 2018 9:52 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello Andrew, On 2018-12-16 00:01, Andrew Luo wrote: For me, /mnt/c was already mounted case insensitive. Maybe this is only the default for the C:\ drive though (or perhaps depends on your Windows/WSL version?) I think the default is "dir", which will cause any new directory created from WSL to be case sensitive, so
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hi Erik, Which target are you using (make exploded-image?)? I never saw this error while building on my machine (I've built about 10 times now, I'm on Windows 10 1809 for what it's worth). Perhaps I can try to reproduce this on my system as well... Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Wednesday, December 19, 2018 8:28 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I'm now seeing intermittent build failures that look like this: make[3]: *** No rule to make target '/mnt/d/erik/jdk-sandbox/open/src/java.security.jgss/share/classes/sun/security/krb5/internal/TGSReq.java', needed by '/mnt/d/erik/jdk-sandbox/build/windows-x86_64-server-release/jdk/modules/java.security.jgss/_the.java.security.jgss_batch'. Stop. The particular file that's missing varies, and deleting the output dir for that module and rebuild works. The common pattern seems to be upper case letters in the file name of the source file. I will investigate some more. /Erik On 2018-12-19 06:18, Erik Joelsson wrote: > I can also report that on the Windows 10 machine I'm testing this on, > "make bundles" on Cygwin took 12m56s and in WSL 8m39s, so a pretty big > improvement! > > /Erik > > On 2018-12-19 03:45, Erik Joelsson wrote: >> Hello, >> >> On 2018-12-19 00:19, Erik Joelsson wrote: >>> Hello Andrew, >>> >>> On 2018-12-18 12:45, Andrew Luo wrote: >>>> Hi Erik/Magnus, >>>> >>>> I've attached my latest changes: >>>> >>>> 1. Fixed a file I forgot to revert in my previous change while >>>> trying something out... >>>> 2. Added information about case sensitivity of the OpenJDK build >>>> directory (yes, I did use the make target to generate the HTML >>>> file) 3. Fixed Cygwin (hopefully, I don't have a Cygwin environment >>>> to verify this) >>> >>> I will take this patch for a spin and see. >>> >> After applying a fix for >> https://bugs.openjdk.java.net/browse/JDK-8215635, I managed to build >> everything as well. I pushed some minor adjustments to make Cygwin >> work too. >> >> I will need to take this through more thorough testing. >> >> /Erik >> >>>> With this patch I've tested the following targets: >>>> exploded-image (default): Works >>>> images: Works >>>> bundles: Works >>>> test: Completes, but some tests fail. >>>> >>>> I didn't go through the test failures completely, but all of the >>>> ones I did see are due to attempting to call CreateProcess with >>>> "sh" as the argument (in jtreg: >>>> http://hg.openjdk.java.net/code-tools/jtreg/file/36c592d2f544/src/share/classes/com/sun/javatest/regtest/exec/ShellAction.java). >>>> >>>> This isn't supported in Windows, perhaps using a Linux boot JDK >>>> would fix this (but might break other things). I can look into >>>> fixing it (on WSL you can call "wsl sh", for example), but I think >>>> since it's a completely separate repo anyways, it would be best to >>>> take up those changes separately. Let me know your thoughts on this. >>> >>> Ah, if a Java process is launched from a Cygwin environment, it will >>> have the unix/cygwin tools in the path so those can be launched >>> directly. When running in WSL, it will launch the Windows binary >>> java.exe in the Windows environment so there is no trace of WSL. I >>> agree that we can look into this later, but we need to note that >>> running tests is not completely supported in WSL. >>> >>> /Erik >>> >>>> Otherwise, let me know if there is any other comments/suggestions >>>> before we can merge this into the main repository. >>>> >>>> Thanks, >>>> >>>> -Andrew >>>> >>>> -Original Message- >>>> From: Erik Joelsson >>>> Sent: Monday, December 17, 2018 9:52 AM >>>> To: Andrew Luo ; Magnus Ihse >>>> Bursie >>>> Cc: build-dev@openjdk.java.net >>>> Subject: Re: [PATCH] Support for building using WSL (Windows >>>> Subsystem for Linux) on Windows >>>> >>>> Hello Andrew, >>>> >>>> On 2018-12-16 00:01, Andrew Luo wrote: >>>>> For me, /mnt/c was already mounted case insensitive. Maybe this >>>>> is only the default for the C:\ d
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
I'm now seeing intermittent build failures that look like this: make[3]: *** No rule to make target '/mnt/d/erik/jdk-sandbox/open/src/java.security.jgss/share/classes/sun/security/krb5/internal/TGSReq.java', needed by '/mnt/d/erik/jdk-sandbox/build/windows-x86_64-server-release/jdk/modules/java.security.jgss/_the.java.security.jgss_batch'. Stop. The particular file that's missing varies, and deleting the output dir for that module and rebuild works. The common pattern seems to be upper case letters in the file name of the source file. I will investigate some more. /Erik On 2018-12-19 06:18, Erik Joelsson wrote: I can also report that on the Windows 10 machine I'm testing this on, "make bundles" on Cygwin took 12m56s and in WSL 8m39s, so a pretty big improvement! /Erik On 2018-12-19 03:45, Erik Joelsson wrote: Hello, On 2018-12-19 00:19, Erik Joelsson wrote: Hello Andrew, On 2018-12-18 12:45, Andrew Luo wrote: Hi Erik/Magnus, I've attached my latest changes: 1. Fixed a file I forgot to revert in my previous change while trying something out... 2. Added information about case sensitivity of the OpenJDK build directory (yes, I did use the make target to generate the HTML file) 3. Fixed Cygwin (hopefully, I don't have a Cygwin environment to verify this) I will take this patch for a spin and see. After applying a fix for https://bugs.openjdk.java.net/browse/JDK-8215635, I managed to build everything as well. I pushed some minor adjustments to make Cygwin work too. I will need to take this through more thorough testing. /Erik With this patch I've tested the following targets: exploded-image (default): Works images: Works bundles: Works test: Completes, but some tests fail. I didn't go through the test failures completely, but all of the ones I did see are due to attempting to call CreateProcess with "sh" as the argument (in jtreg: http://hg.openjdk.java.net/code-tools/jtreg/file/36c592d2f544/src/share/classes/com/sun/javatest/regtest/exec/ShellAction.java). This isn't supported in Windows, perhaps using a Linux boot JDK would fix this (but might break other things). I can look into fixing it (on WSL you can call "wsl sh", for example), but I think since it's a completely separate repo anyways, it would be best to take up those changes separately. Let me know your thoughts on this. Ah, if a Java process is launched from a Cygwin environment, it will have the unix/cygwin tools in the path so those can be launched directly. When running in WSL, it will launch the Windows binary java.exe in the Windows environment so there is no trace of WSL. I agree that we can look into this later, but we need to note that running tests is not completely supported in WSL. /Erik Otherwise, let me know if there is any other comments/suggestions before we can merge this into the main repository. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 17, 2018 9:52 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello Andrew, On 2018-12-16 00:01, Andrew Luo wrote: For me, /mnt/c was already mounted case insensitive. Maybe this is only the default for the C:\ drive though (or perhaps depends on your Windows/WSL version?) I think the default is "dir", which will cause any new directory created from WSL to be case sensitive, so I would say this needs to be documented in building.md. Anyways, I've synced down the sandbox and added a new patch to address some of the feedback (and some of my own minor enhancements): 1. Got rid of EXECUTABLE_SUFFIX in favor of EXE_SUFFIX, which had to move earlier in the sequence 2. Use $EXE_SUFFIX instead of .exe literal per Magnus' feedback 3. Added information about WSL versioning to build, similar to Cygwin 4. Updated building.md and building.html with WSL build instructions Nice! I've applied and pushed this patch to the sandbox. Just to be sure, did you generate the html version with pandoc using "make update-build-docs"? If not, we will need to make sure that is done before the final push. I noticed trailing whitespace in some files. Jcheck will reject that in most types of files but in the build, we are a bit on our own trying to avoid it. (By the way, you misspelled my name in your sandbox commit): http://hg.openjdk.java.net/jdk/sandbox/rev/12615de8335e Terribly sorry about that! The combination of u and o is a common slip for me on the keyboard. It's correct in the new commit at least, and in the final commit, contributors are attributed with email addresses. I will work on fixing the Cygwin path extraction in my next patch. Most likely I will just restore the old code for Cygwin while using the new code for WSL, unless there are other suggestions... Aside from this, is there any other feedback th
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
I can also report that on the Windows 10 machine I'm testing this on, "make bundles" on Cygwin took 12m56s and in WSL 8m39s, so a pretty big improvement! /Erik On 2018-12-19 03:45, Erik Joelsson wrote: Hello, On 2018-12-19 00:19, Erik Joelsson wrote: Hello Andrew, On 2018-12-18 12:45, Andrew Luo wrote: Hi Erik/Magnus, I've attached my latest changes: 1. Fixed a file I forgot to revert in my previous change while trying something out... 2. Added information about case sensitivity of the OpenJDK build directory (yes, I did use the make target to generate the HTML file) 3. Fixed Cygwin (hopefully, I don't have a Cygwin environment to verify this) I will take this patch for a spin and see. After applying a fix for https://bugs.openjdk.java.net/browse/JDK-8215635, I managed to build everything as well. I pushed some minor adjustments to make Cygwin work too. I will need to take this through more thorough testing. /Erik With this patch I've tested the following targets: exploded-image (default): Works images: Works bundles: Works test: Completes, but some tests fail. I didn't go through the test failures completely, but all of the ones I did see are due to attempting to call CreateProcess with "sh" as the argument (in jtreg: http://hg.openjdk.java.net/code-tools/jtreg/file/36c592d2f544/src/share/classes/com/sun/javatest/regtest/exec/ShellAction.java). This isn't supported in Windows, perhaps using a Linux boot JDK would fix this (but might break other things). I can look into fixing it (on WSL you can call "wsl sh", for example), but I think since it's a completely separate repo anyways, it would be best to take up those changes separately. Let me know your thoughts on this. Ah, if a Java process is launched from a Cygwin environment, it will have the unix/cygwin tools in the path so those can be launched directly. When running in WSL, it will launch the Windows binary java.exe in the Windows environment so there is no trace of WSL. I agree that we can look into this later, but we need to note that running tests is not completely supported in WSL. /Erik Otherwise, let me know if there is any other comments/suggestions before we can merge this into the main repository. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 17, 2018 9:52 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello Andrew, On 2018-12-16 00:01, Andrew Luo wrote: For me, /mnt/c was already mounted case insensitive. Maybe this is only the default for the C:\ drive though (or perhaps depends on your Windows/WSL version?) I think the default is "dir", which will cause any new directory created from WSL to be case sensitive, so I would say this needs to be documented in building.md. Anyways, I've synced down the sandbox and added a new patch to address some of the feedback (and some of my own minor enhancements): 1. Got rid of EXECUTABLE_SUFFIX in favor of EXE_SUFFIX, which had to move earlier in the sequence 2. Use $EXE_SUFFIX instead of .exe literal per Magnus' feedback 3. Added information about WSL versioning to build, similar to Cygwin 4. Updated building.md and building.html with WSL build instructions Nice! I've applied and pushed this patch to the sandbox. Just to be sure, did you generate the html version with pandoc using "make update-build-docs"? If not, we will need to make sure that is done before the final push. I noticed trailing whitespace in some files. Jcheck will reject that in most types of files but in the build, we are a bit on our own trying to avoid it. (By the way, you misspelled my name in your sandbox commit): http://hg.openjdk.java.net/jdk/sandbox/rev/12615de8335e Terribly sorry about that! The combination of u and o is a common slip for me on the keyboard. It's correct in the new commit at least, and in the final commit, contributors are attributed with email addresses. I will work on fixing the Cygwin path extraction in my next patch. Most likely I will just restore the old code for Cygwin while using the new code for WSL, unless there are other suggestions... Aside from this, is there any other feedback that I should take into account before we can merge this into the main repository? That may be the best solution. I tried to build some more targets and failed. Please make sure you can do "make bundles". That will build docs and tests in addition to just the product and also do the packaging into zip/tar.gz. It would also be nice if "make test" worked. Note that Magnus is now on vacation and I will be traveling, so you will not hear from me until Wednesday. /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 5:42 PM To: Andrew Luo ; Magnus Ihse Bursie Cc: buil
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hello, On 2018-12-19 00:19, Erik Joelsson wrote: Hello Andrew, On 2018-12-18 12:45, Andrew Luo wrote: Hi Erik/Magnus, I've attached my latest changes: 1. Fixed a file I forgot to revert in my previous change while trying something out... 2. Added information about case sensitivity of the OpenJDK build directory (yes, I did use the make target to generate the HTML file) 3. Fixed Cygwin (hopefully, I don't have a Cygwin environment to verify this) I will take this patch for a spin and see. After applying a fix for https://bugs.openjdk.java.net/browse/JDK-8215635, I managed to build everything as well. I pushed some minor adjustments to make Cygwin work too. I will need to take this through more thorough testing. /Erik With this patch I've tested the following targets: exploded-image (default): Works images: Works bundles: Works test: Completes, but some tests fail. I didn't go through the test failures completely, but all of the ones I did see are due to attempting to call CreateProcess with "sh" as the argument (in jtreg: http://hg.openjdk.java.net/code-tools/jtreg/file/36c592d2f544/src/share/classes/com/sun/javatest/regtest/exec/ShellAction.java). This isn't supported in Windows, perhaps using a Linux boot JDK would fix this (but might break other things). I can look into fixing it (on WSL you can call "wsl sh", for example), but I think since it's a completely separate repo anyways, it would be best to take up those changes separately. Let me know your thoughts on this. Ah, if a Java process is launched from a Cygwin environment, it will have the unix/cygwin tools in the path so those can be launched directly. When running in WSL, it will launch the Windows binary java.exe in the Windows environment so there is no trace of WSL. I agree that we can look into this later, but we need to note that running tests is not completely supported in WSL. /Erik Otherwise, let me know if there is any other comments/suggestions before we can merge this into the main repository. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 17, 2018 9:52 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello Andrew, On 2018-12-16 00:01, Andrew Luo wrote: For me, /mnt/c was already mounted case insensitive. Maybe this is only the default for the C:\ drive though (or perhaps depends on your Windows/WSL version?) I think the default is "dir", which will cause any new directory created from WSL to be case sensitive, so I would say this needs to be documented in building.md. Anyways, I've synced down the sandbox and added a new patch to address some of the feedback (and some of my own minor enhancements): 1. Got rid of EXECUTABLE_SUFFIX in favor of EXE_SUFFIX, which had to move earlier in the sequence 2. Use $EXE_SUFFIX instead of .exe literal per Magnus' feedback 3. Added information about WSL versioning to build, similar to Cygwin 4. Updated building.md and building.html with WSL build instructions Nice! I've applied and pushed this patch to the sandbox. Just to be sure, did you generate the html version with pandoc using "make update-build-docs"? If not, we will need to make sure that is done before the final push. I noticed trailing whitespace in some files. Jcheck will reject that in most types of files but in the build, we are a bit on our own trying to avoid it. (By the way, you misspelled my name in your sandbox commit): http://hg.openjdk.java.net/jdk/sandbox/rev/12615de8335e Terribly sorry about that! The combination of u and o is a common slip for me on the keyboard. It's correct in the new commit at least, and in the final commit, contributors are attributed with email addresses. I will work on fixing the Cygwin path extraction in my next patch. Most likely I will just restore the old code for Cygwin while using the new code for WSL, unless there are other suggestions... Aside from this, is there any other feedback that I should take into account before we can merge this into the main repository? That may be the best solution. I tried to build some more targets and failed. Please make sure you can do "make bundles". That will build docs and tests in addition to just the product and also do the packaging into zip/tar.gz. It would also be nice if "make test" worked. Note that Magnus is now on vacation and I will be traveling, so you will not hear from me until Wednesday. /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 5:42 PM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows After having configured my WSL to mount using case=off, I was able to successfully build image
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hello Andrew, On 2018-12-18 12:45, Andrew Luo wrote: Hi Erik/Magnus, I've attached my latest changes: 1. Fixed a file I forgot to revert in my previous change while trying something out... 2. Added information about case sensitivity of the OpenJDK build directory (yes, I did use the make target to generate the HTML file) 3. Fixed Cygwin (hopefully, I don't have a Cygwin environment to verify this) I will take this patch for a spin and see. With this patch I've tested the following targets: exploded-image (default): Works images: Works bundles: Works test: Completes, but some tests fail. I didn't go through the test failures completely, but all of the ones I did see are due to attempting to call CreateProcess with "sh" as the argument (in jtreg: http://hg.openjdk.java.net/code-tools/jtreg/file/36c592d2f544/src/share/classes/com/sun/javatest/regtest/exec/ShellAction.java). This isn't supported in Windows, perhaps using a Linux boot JDK would fix this (but might break other things). I can look into fixing it (on WSL you can call "wsl sh", for example), but I think since it's a completely separate repo anyways, it would be best to take up those changes separately. Let me know your thoughts on this. Ah, if a Java process is launched from a Cygwin environment, it will have the unix/cygwin tools in the path so those can be launched directly. When running in WSL, it will launch the Windows binary java.exe in the Windows environment so there is no trace of WSL. I agree that we can look into this later, but we need to note that running tests is not completely supported in WSL. /Erik Otherwise, let me know if there is any other comments/suggestions before we can merge this into the main repository. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 17, 2018 9:52 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello Andrew, On 2018-12-16 00:01, Andrew Luo wrote: For me, /mnt/c was already mounted case insensitive. Maybe this is only the default for the C:\ drive though (or perhaps depends on your Windows/WSL version?) I think the default is "dir", which will cause any new directory created from WSL to be case sensitive, so I would say this needs to be documented in building.md. Anyways, I've synced down the sandbox and added a new patch to address some of the feedback (and some of my own minor enhancements): 1. Got rid of EXECUTABLE_SUFFIX in favor of EXE_SUFFIX, which had to move earlier in the sequence 2. Use $EXE_SUFFIX instead of .exe literal per Magnus' feedback 3. Added information about WSL versioning to build, similar to Cygwin 4. Updated building.md and building.html with WSL build instructions Nice! I've applied and pushed this patch to the sandbox. Just to be sure, did you generate the html version with pandoc using "make update-build-docs"? If not, we will need to make sure that is done before the final push. I noticed trailing whitespace in some files. Jcheck will reject that in most types of files but in the build, we are a bit on our own trying to avoid it. (By the way, you misspelled my name in your sandbox commit): http://hg.openjdk.java.net/jdk/sandbox/rev/12615de8335e Terribly sorry about that! The combination of u and o is a common slip for me on the keyboard. It's correct in the new commit at least, and in the final commit, contributors are attributed with email addresses. I will work on fixing the Cygwin path extraction in my next patch. Most likely I will just restore the old code for Cygwin while using the new code for WSL, unless there are other suggestions... Aside from this, is there any other feedback that I should take into account before we can merge this into the main repository? That may be the best solution. I tried to build some more targets and failed. Please make sure you can do "make bundles". That will build docs and tests in addition to just the product and also do the packaging into zip/tar.gz. It would also be nice if "make test" worked. Note that Magnus is now on vacation and I will be traveling, so you will not hear from me until Wednesday. /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 5:42 PM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows After having configured my WSL to mount using case=off, I was able to successfully build images using the latest patch as applied in the sandbox. /Erik On 2018-12-14 17:23, Erik Joelsson wrote: Hello again, I took the liberty of creating a bug for this and also a sandbox branch where I've applied your latest patch. If you clone that you can send further patches based on a known st
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hi Erik/Magnus, I've attached my latest changes: 1. Fixed a file I forgot to revert in my previous change while trying something out... 2. Added information about case sensitivity of the OpenJDK build directory (yes, I did use the make target to generate the HTML file) 3. Fixed Cygwin (hopefully, I don't have a Cygwin environment to verify this) With this patch I've tested the following targets: exploded-image (default): Works images: Works bundles: Works test: Completes, but some tests fail. I didn't go through the test failures completely, but all of the ones I did see are due to attempting to call CreateProcess with "sh" as the argument (in jtreg: http://hg.openjdk.java.net/code-tools/jtreg/file/36c592d2f544/src/share/classes/com/sun/javatest/regtest/exec/ShellAction.java). This isn't supported in Windows, perhaps using a Linux boot JDK would fix this (but might break other things). I can look into fixing it (on WSL you can call "wsl sh", for example), but I think since it's a completely separate repo anyways, it would be best to take up those changes separately. Let me know your thoughts on this. Otherwise, let me know if there is any other comments/suggestions before we can merge this into the main repository. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 17, 2018 9:52 AM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello Andrew, On 2018-12-16 00:01, Andrew Luo wrote: > For me, /mnt/c was already mounted case insensitive. Maybe this is > only the default for the C:\ drive though (or perhaps depends on your > Windows/WSL version?) I think the default is "dir", which will cause any new directory created from WSL to be case sensitive, so I would say this needs to be documented in building.md. > Anyways, I've synced down the sandbox and added a new patch to address some > of the feedback (and some of my own minor enhancements): > > 1. Got rid of EXECUTABLE_SUFFIX in favor of EXE_SUFFIX, which had to move > earlier in the sequence > 2. Use $EXE_SUFFIX instead of .exe literal per Magnus' feedback > 3. Added information about WSL versioning to build, similar to Cygwin > 4. Updated building.md and building.html with WSL build instructions Nice! I've applied and pushed this patch to the sandbox. Just to be sure, did you generate the html version with pandoc using "make update-build-docs"? If not, we will need to make sure that is done before the final push. I noticed trailing whitespace in some files. Jcheck will reject that in most types of files but in the build, we are a bit on our own trying to avoid it. > > (By the way, you misspelled my name in your sandbox commit): > http://hg.openjdk.java.net/jdk/sandbox/rev/12615de8335e Terribly sorry about that! The combination of u and o is a common slip for me on the keyboard. It's correct in the new commit at least, and in the final commit, contributors are attributed with email addresses. > > I will work on fixing the Cygwin path extraction in my next patch. Most > likely I will just restore the old code for Cygwin while using the new code > for WSL, unless there are other suggestions... Aside from this, is there any > other feedback that I should take into account before we can merge this into > the main repository? That may be the best solution. I tried to build some more targets and failed. Please make sure you can do "make bundles". That will build docs and tests in addition to just the product and also do the packaging into zip/tar.gz. It would also be nice if "make test" worked. Note that Magnus is now on vacation and I will be traveling, so you will not hear from me until Wednesday. /Erik > > Thanks, > > -Andrew > > -Original Message- > From: Erik Joelsson > Sent: Friday, December 14, 2018 5:42 PM > To: Andrew Luo ; Magnus Ihse Bursie > > Cc: build-dev@openjdk.java.net > Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for > Linux) on Windows > > After having configured my WSL to mount using case=off, I was able to > successfully build images using the latest patch as applied in the sandbox. > > /Erik > > On 2018-12-14 17:23, Erik Joelsson wrote: >> Hello again, >> >> I took the liberty of creating a bug for this and also a sandbox >> branch where I've applied your latest patch. If you clone that you can >> send further patches based on a known state in the sandbox. This will >> make it easier to see what you are actually doing in each update, as >> well as give us better references when discussing them. It also gives >> me the ability to directly change things so we can keep Cygwin/m
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hello Andrew, On 2018-12-16 00:01, Andrew Luo wrote: For me, /mnt/c was already mounted case insensitive. Maybe this is only the default for the C:\ drive though (or perhaps depends on your Windows/WSL version?) I think the default is "dir", which will cause any new directory created from WSL to be case sensitive, so I would say this needs to be documented in building.md. Anyways, I've synced down the sandbox and added a new patch to address some of the feedback (and some of my own minor enhancements): 1. Got rid of EXECUTABLE_SUFFIX in favor of EXE_SUFFIX, which had to move earlier in the sequence 2. Use $EXE_SUFFIX instead of .exe literal per Magnus' feedback 3. Added information about WSL versioning to build, similar to Cygwin 4. Updated building.md and building.html with WSL build instructions Nice! I've applied and pushed this patch to the sandbox. Just to be sure, did you generate the html version with pandoc using "make update-build-docs"? If not, we will need to make sure that is done before the final push. I noticed trailing whitespace in some files. Jcheck will reject that in most types of files but in the build, we are a bit on our own trying to avoid it. (By the way, you misspelled my name in your sandbox commit): http://hg.openjdk.java.net/jdk/sandbox/rev/12615de8335e Terribly sorry about that! The combination of u and o is a common slip for me on the keyboard. It's correct in the new commit at least, and in the final commit, contributors are attributed with email addresses. I will work on fixing the Cygwin path extraction in my next patch. Most likely I will just restore the old code for Cygwin while using the new code for WSL, unless there are other suggestions... Aside from this, is there any other feedback that I should take into account before we can merge this into the main repository? That may be the best solution. I tried to build some more targets and failed. Please make sure you can do "make bundles". That will build docs and tests in addition to just the product and also do the packaging into zip/tar.gz. It would also be nice if "make test" worked. Note that Magnus is now on vacation and I will be traveling, so you will not hear from me until Wednesday. /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 5:42 PM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows After having configured my WSL to mount using case=off, I was able to successfully build images using the latest patch as applied in the sandbox. /Erik On 2018-12-14 17:23, Erik Joelsson wrote: Hello again, I took the liberty of creating a bug for this and also a sandbox branch where I've applied your latest patch. If you clone that you can send further patches based on a known state in the sandbox. This will make it easier to see what you are actually doing in each update, as well as give us better references when discussing them. It also gives me the ability to directly change things so we can keep Cygwin/msys working. https://bugs.openjdk.java.net/browse/JDK-8215445 http://hg.openjdk.java.net/jdk/sandbox/shortlog/12615de8335e /Erik On 2018-12-14 16:47, Erik Joelsson wrote: Hello, You beat me to it. I just found the rc.exe problem was that FIXPATH_PATH in spec.gmk.in was quoted. Make just propagates quotes verbatim, so then fixpath.c would create a path variable like; $PATH;"$FIXPATH_PATH" Which is why link.exe could not find rc.exe. /Erik On 2018-12-14 16:32, Andrew Luo wrote: Ok, here's my latest patch (I didn't add your case sensitivity fix yet, but will do next patch). I believe this should get you past the rc.exe issues. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 4:15 PM To: Magnus Ihse Bursie Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 16:06, Magnus Ihse Bursie wrote: 14 dec. 2018 kl. 23:42 skrev Erik Joelsson : I found the reason it's not failing make. It returns "1" and NativeCompilation.gmk currently ignores 1 explicitly for Visual Studio. I added that back in 2014 in https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't figure out why. Nothing mentioned in either comment or review. Sounds like it's ripe for removal then. :) I wonder what kind of issue you might have run into that caused a returned 1 to happen and yet we didn't want to consider it a failure... If I'm to guess, I think it's one of the commands we pipe the output to when the output is zero. This would explain why it was added together with pipefail. /Erik /Magnus /Erik On 2018-12-14 13:59, Magnus Ihse Bursie wrote: On 2018-12-14 22:15, Erik Joelsson wrote: I get the same erro
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
For me, /mnt/c was already mounted case insensitive. Maybe this is only the default for the C:\ drive though (or perhaps depends on your Windows/WSL version?) Anyways, I've synced down the sandbox and added a new patch to address some of the feedback (and some of my own minor enhancements): 1. Got rid of EXECUTABLE_SUFFIX in favor of EXE_SUFFIX, which had to move earlier in the sequence 2. Use $EXE_SUFFIX instead of .exe literal per Magnus' feedback 3. Added information about WSL versioning to build, similar to Cygwin 4. Updated building.md and building.html with WSL build instructions (By the way, you misspelled my name in your sandbox commit): http://hg.openjdk.java.net/jdk/sandbox/rev/12615de8335e I will work on fixing the Cygwin path extraction in my next patch. Most likely I will just restore the old code for Cygwin while using the new code for WSL, unless there are other suggestions... Aside from this, is there any other feedback that I should take into account before we can merge this into the main repository? Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 5:42 PM To: Andrew Luo ; Magnus Ihse Bursie Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows After having configured my WSL to mount using case=off, I was able to successfully build images using the latest patch as applied in the sandbox. /Erik On 2018-12-14 17:23, Erik Joelsson wrote: > Hello again, > > I took the liberty of creating a bug for this and also a sandbox > branch where I've applied your latest patch. If you clone that you can > send further patches based on a known state in the sandbox. This will > make it easier to see what you are actually doing in each update, as > well as give us better references when discussing them. It also gives > me the ability to directly change things so we can keep Cygwin/msys > working. > > https://bugs.openjdk.java.net/browse/JDK-8215445 > > http://hg.openjdk.java.net/jdk/sandbox/shortlog/12615de8335e > > /Erik > > On 2018-12-14 16:47, Erik Joelsson wrote: >> Hello, >> >> You beat me to it. I just found the rc.exe problem was that >> FIXPATH_PATH in spec.gmk.in was quoted. Make just propagates quotes >> verbatim, so then fixpath.c would create a path variable like; >> >> $PATH;"$FIXPATH_PATH" >> >> Which is why link.exe could not find rc.exe. >> >> /Erik >> >> On 2018-12-14 16:32, Andrew Luo wrote: >>> Ok, here's my latest patch (I didn't add your case sensitivity fix >>> yet, but will do next patch). I believe this should get you past >>> the rc.exe issues. >>> >>> Thanks, >>> >>> -Andrew >>> >>> -----Original Message- >>> From: Erik Joelsson >>> Sent: Friday, December 14, 2018 4:15 PM >>> To: Magnus Ihse Bursie >>> Cc: Andrew Luo ; >>> build-dev@openjdk.java.net >>> Subject: Re: [PATCH] Support for building using WSL (Windows >>> Subsystem for Linux) on Windows >>> >>> >>> On 2018-12-14 16:06, Magnus Ihse Bursie wrote: >>>>> 14 dec. 2018 kl. 23:42 skrev Erik Joelsson >>>>> : >>>>> >>>>> I found the reason it's not failing make. It returns "1" and >>>>> NativeCompilation.gmk currently ignores 1 explicitly for Visual >>>>> Studio. I added that back in 2014 in >>>>> https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't >>>>> figure out why. Nothing mentioned in either comment or review. >>>> Sounds like it's ripe for removal then. :) I wonder what kind of >>>> issue you might have run into that caused a returned 1 to happen >>>> and yet we didn't want to consider it a failure... >>> If I'm to guess, I think it's one of the commands we pipe the output >>> to when the output is zero. This would explain why it was added >>> together with pipefail. >>> >>> /Erik >>> >>>> /Magnus >>>> >>>>> /Erik >>>>> >>>>>> On 2018-12-14 13:59, Magnus Ihse Bursie wrote: >>>>>> >>>>>> >>>>>>> On 2018-12-14 22:15, Erik Joelsson wrote: >>>>>>> I get the same error for pch and it still continues, but this >>>>>>> time I let it run until it eventually fails for real when it >>>>>>> can't link. Perhaps it's simply cl.exe that isn't returning non >>>>>>> zero for this error? When the linker fai
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
After having configured my WSL to mount using case=off, I was able to successfully build images using the latest patch as applied in the sandbox. /Erik On 2018-12-14 17:23, Erik Joelsson wrote: Hello again, I took the liberty of creating a bug for this and also a sandbox branch where I've applied your latest patch. If you clone that you can send further patches based on a known state in the sandbox. This will make it easier to see what you are actually doing in each update, as well as give us better references when discussing them. It also gives me the ability to directly change things so we can keep Cygwin/msys working. https://bugs.openjdk.java.net/browse/JDK-8215445 http://hg.openjdk.java.net/jdk/sandbox/shortlog/12615de8335e /Erik On 2018-12-14 16:47, Erik Joelsson wrote: Hello, You beat me to it. I just found the rc.exe problem was that FIXPATH_PATH in spec.gmk.in was quoted. Make just propagates quotes verbatim, so then fixpath.c would create a path variable like; $PATH;"$FIXPATH_PATH" Which is why link.exe could not find rc.exe. /Erik On 2018-12-14 16:32, Andrew Luo wrote: Ok, here's my latest patch (I didn't add your case sensitivity fix yet, but will do next patch). I believe this should get you past the rc.exe issues. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 4:15 PM To: Magnus Ihse Bursie Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 16:06, Magnus Ihse Bursie wrote: 14 dec. 2018 kl. 23:42 skrev Erik Joelsson : I found the reason it's not failing make. It returns "1" and NativeCompilation.gmk currently ignores 1 explicitly for Visual Studio. I added that back in 2014 in https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't figure out why. Nothing mentioned in either comment or review. Sounds like it's ripe for removal then. :) I wonder what kind of issue you might have run into that caused a returned 1 to happen and yet we didn't want to consider it a failure... If I'm to guess, I think it's one of the commands we pipe the output to when the output is zero. This would explain why it was added together with pipefail. /Erik /Magnus /Erik On 2018-12-14 13:59, Magnus Ihse Bursie wrote: On 2018-12-14 22:15, Erik Joelsson wrote: I get the same error for pch and it still continues, but this time I let it run until it eventually fails for real when it can't link. Perhaps it's simply cl.exe that isn't returning non zero for this error? When the linker fails, make fails, so propagation doesn't seem broken. That does also seem really weird, considering that it claims it to be a "fatal error". Can you repeat the command at the command line and get the failure again, and then check the return value? Can you rewrite the command line and run it from the devenv prompt? That is, is there any indication that the pch file itself is messed up, or can it be used if running the compilation that should use it from an "ok" prompt? /Magnus /Erik On 2018-12-14 12:55, Andrew Luo wrote: Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating... Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Friday, December 14, 2018 12:34 PM To: 'Andrew Luo' ; Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : On 2018-12-14 11:05, Magnus Ihse Bursie wrote: On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\ varia nt-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, wh
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hello again, I took the liberty of creating a bug for this and also a sandbox branch where I've applied your latest patch. If you clone that you can send further patches based on a known state in the sandbox. This will make it easier to see what you are actually doing in each update, as well as give us better references when discussing them. It also gives me the ability to directly change things so we can keep Cygwin/msys working. https://bugs.openjdk.java.net/browse/JDK-8215445 http://hg.openjdk.java.net/jdk/sandbox/shortlog/12615de8335e /Erik On 2018-12-14 16:47, Erik Joelsson wrote: Hello, You beat me to it. I just found the rc.exe problem was that FIXPATH_PATH in spec.gmk.in was quoted. Make just propagates quotes verbatim, so then fixpath.c would create a path variable like; $PATH;"$FIXPATH_PATH" Which is why link.exe could not find rc.exe. /Erik On 2018-12-14 16:32, Andrew Luo wrote: Ok, here's my latest patch (I didn't add your case sensitivity fix yet, but will do next patch). I believe this should get you past the rc.exe issues. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 4:15 PM To: Magnus Ihse Bursie Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 16:06, Magnus Ihse Bursie wrote: 14 dec. 2018 kl. 23:42 skrev Erik Joelsson : I found the reason it's not failing make. It returns "1" and NativeCompilation.gmk currently ignores 1 explicitly for Visual Studio. I added that back in 2014 in https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't figure out why. Nothing mentioned in either comment or review. Sounds like it's ripe for removal then. :) I wonder what kind of issue you might have run into that caused a returned 1 to happen and yet we didn't want to consider it a failure... If I'm to guess, I think it's one of the commands we pipe the output to when the output is zero. This would explain why it was added together with pipefail. /Erik /Magnus /Erik On 2018-12-14 13:59, Magnus Ihse Bursie wrote: On 2018-12-14 22:15, Erik Joelsson wrote: I get the same error for pch and it still continues, but this time I let it run until it eventually fails for real when it can't link. Perhaps it's simply cl.exe that isn't returning non zero for this error? When the linker fails, make fails, so propagation doesn't seem broken. That does also seem really weird, considering that it claims it to be a "fatal error". Can you repeat the command at the command line and get the failure again, and then check the return value? Can you rewrite the command line and run it from the devenv prompt? That is, is there any indication that the pch file itself is messed up, or can it be used if running the compilation that should use it from an "ok" prompt? /Magnus /Erik On 2018-12-14 12:55, Andrew Luo wrote: Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating... Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Friday, December 14, 2018 12:34 PM To: 'Andrew Luo' ; Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : On 2018-12-14 11:05, Magnus Ihse Bursie wrote: On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\ varia nt-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Could you? That is, is it o
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Volume Serial Number is 4ED4-C471 Directory of d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs 2018-12-14 14:4136,634,624 BUILD_LIBJVM.pch 1 File(s) 36,634,624 bytes 0 Dir(s) 192,267,493,376 bytes free --- /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 10:42 AM To: Magnus Ihse Bursie ; Andrew Luo Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe -w /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ -wd4800 -WX -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- "-DTHIS_FILE=\"\"" -c -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles/ad_x86_expand.cpp< fixpath using wsl mode, with path list: fixpath converted line c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -Id:/erik/jdk-wsl/closed/src/hotspot/share -Id:/erik/jdk-wsl/open/src/hotspot/share -Id:/erik/jdk-wsl/open/src/hotspot/os/windows -Id:/erik/jdk-ws
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hello, You beat me to it. I just found the rc.exe problem was that FIXPATH_PATH in spec.gmk.in was quoted. Make just propagates quotes verbatim, so then fixpath.c would create a path variable like; $PATH;"$FIXPATH_PATH" Which is why link.exe could not find rc.exe. /Erik On 2018-12-14 16:32, Andrew Luo wrote: Ok, here's my latest patch (I didn't add your case sensitivity fix yet, but will do next patch). I believe this should get you past the rc.exe issues. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 4:15 PM To: Magnus Ihse Bursie Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 16:06, Magnus Ihse Bursie wrote: 14 dec. 2018 kl. 23:42 skrev Erik Joelsson : I found the reason it's not failing make. It returns "1" and NativeCompilation.gmk currently ignores 1 explicitly for Visual Studio. I added that back in 2014 in https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't figure out why. Nothing mentioned in either comment or review. Sounds like it's ripe for removal then. :) I wonder what kind of issue you might have run into that caused a returned 1 to happen and yet we didn't want to consider it a failure... If I'm to guess, I think it's one of the commands we pipe the output to when the output is zero. This would explain why it was added together with pipefail. /Erik /Magnus /Erik On 2018-12-14 13:59, Magnus Ihse Bursie wrote: On 2018-12-14 22:15, Erik Joelsson wrote: I get the same error for pch and it still continues, but this time I let it run until it eventually fails for real when it can't link. Perhaps it's simply cl.exe that isn't returning non zero for this error? When the linker fails, make fails, so propagation doesn't seem broken. That does also seem really weird, considering that it claims it to be a "fatal error". Can you repeat the command at the command line and get the failure again, and then check the return value? Can you rewrite the command line and run it from the devenv prompt? That is, is there any indication that the pch file itself is messed up, or can it be used if running the compilation that should use it from an "ok" prompt? /Magnus /Erik On 2018-12-14 12:55, Andrew Luo wrote: Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating... Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Friday, December 14, 2018 12:34 PM To: 'Andrew Luo' ; Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : On 2018-12-14 11:05, Magnus Ihse Bursie wrote: On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\ varia nt-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Could you? That is, is it only the PCH that is problematic? Trying that now. Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Weird. What if you, after a failed build, rename it to build_libjvm.pch? Doing that causes a new error: d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. cpp : fatal error C1382: the PCH file 'd:\erik\jdk-wsl\build\windows-
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Ok, here's my latest patch (I didn't add your case sensitivity fix yet, but will do next patch). I believe this should get you past the rc.exe issues. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 4:15 PM To: Magnus Ihse Bursie Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 16:06, Magnus Ihse Bursie wrote: >> 14 dec. 2018 kl. 23:42 skrev Erik Joelsson : >> >> I found the reason it's not failing make. It returns "1" and >> NativeCompilation.gmk currently ignores 1 explicitly for Visual Studio. I >> added that back in 2014 in https://bugs.openjdk.java.net/browse/JDK-8065576, >> but I can't figure out why. Nothing mentioned in either comment or review. > Sounds like it's ripe for removal then. :) I wonder what kind of issue you > might have run into that caused a returned 1 to happen and yet we didn't want > to consider it a failure... If I'm to guess, I think it's one of the commands we pipe the output to when the output is zero. This would explain why it was added together with pipefail. /Erik > /Magnus > >> /Erik >> >>> On 2018-12-14 13:59, Magnus Ihse Bursie wrote: >>> >>> >>>> On 2018-12-14 22:15, Erik Joelsson wrote: >>>> I get the same error for pch and it still continues, but this time I let >>>> it run until it eventually fails for real when it can't link. Perhaps it's >>>> simply cl.exe that isn't returning non zero for this error? When the >>>> linker fails, make fails, so propagation doesn't seem broken. >>> That does also seem really weird, considering that it claims it to be a >>> "fatal error". Can you repeat the command at the command line and get the >>> failure again, and then check the return value? Can you rewrite the command >>> line and run it from the devenv prompt? That is, is there any indication >>> that the pch file itself is messed up, or can it be used if running the >>> compilation that should use it from an "ok" prompt? >>> >>> /Magnus >>>> /Erik >>>> >>>>> On 2018-12-14 12:55, Andrew Luo wrote: >>>>> Hmm, I get the rc.exe error as well, but now it is much later down the >>>>> line... Still investigating... >>>>> >>>>> Thanks, >>>>> >>>>> -Andrew >>>>> >>>>> -Original Message- >>>>> From: Andrew Luo >>>>> Sent: Friday, December 14, 2018 12:34 PM >>>>> To: 'Andrew Luo' ; Magnus Ihse >>>>> Bursie ; Erik Joelsson >>>>> >>>>> Cc: build-dev@openjdk.java.net >>>>> Subject: RE: [PATCH] Support for building using WSL (Windows >>>>> Subsystem for Linux) on Windows >>>>> >>>>> Try this updated patch with some fixes... >>>>> >>>>> Thanks, >>>>> >>>>> -Andrew >>>>> >>>>> -Original Message- >>>>> From: build-dev On Behalf Of >>>>> Andrew Luo >>>>> Sent: Friday, December 14, 2018 12:01 PM >>>>> To: Magnus Ihse Bursie ; Erik >>>>> Joelsson >>>>> Cc: build-dev@openjdk.java.net >>>>> Subject: RE: [PATCH] Support for building using WSL (Windows >>>>> Subsystem for Linux) on Windows >>>>> >>>>> I think I have a fix for it. Give me a minute (or a few hours depending >>>>> on if it works). >>>>> >>>>> Thanks, >>>>> >>>>> -Andrew >>>>> >>>>> -Original Message- >>>>> From: Magnus Ihse Bursie >>>>> Sent: Friday, December 14, 2018 11:42 AM >>>>> To: Erik Joelsson >>>>> Cc: Andrew Luo ; >>>>> build-dev@openjdk.java.net >>>>> Subject: Re: [PATCH] Support for building using WSL (Windows >>>>> Subsystem for Linux) on Windows >>>>> >>>>> >>>>>> 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : >>>>>> >>>>>> >>>>>>> On 2018-12-14 11:05, Magnus Ihse Bursie wrote: >>>>>>> >>>>>>> >>>>>>>>> On 2018-12-14 19:41, Erik Joelsson wrote: >>>>>>>>> >>>>>>>>> On 2018-12-14 10:28, Magnus Ihs
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 16:06, Magnus Ihse Bursie wrote: 14 dec. 2018 kl. 23:42 skrev Erik Joelsson : I found the reason it's not failing make. It returns "1" and NativeCompilation.gmk currently ignores 1 explicitly for Visual Studio. I added that back in 2014 in https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't figure out why. Nothing mentioned in either comment or review. Sounds like it's ripe for removal then. :) I wonder what kind of issue you might have run into that caused a returned 1 to happen and yet we didn't want to consider it a failure... If I'm to guess, I think it's one of the commands we pipe the output to when the output is zero. This would explain why it was added together with pipefail. /Erik /Magnus /Erik On 2018-12-14 13:59, Magnus Ihse Bursie wrote: On 2018-12-14 22:15, Erik Joelsson wrote: I get the same error for pch and it still continues, but this time I let it run until it eventually fails for real when it can't link. Perhaps it's simply cl.exe that isn't returning non zero for this error? When the linker fails, make fails, so propagation doesn't seem broken. That does also seem really weird, considering that it claims it to be a "fatal error". Can you repeat the command at the command line and get the failure again, and then check the return value? Can you rewrite the command line and run it from the devenv prompt? That is, is there any indication that the pch file itself is messed up, or can it be used if running the compilation that should use it from an "ok" prompt? /Magnus /Erik On 2018-12-14 12:55, Andrew Luo wrote: Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating... Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Friday, December 14, 2018 12:34 PM To: 'Andrew Luo' ; Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : On 2018-12-14 11:05, Magnus Ihse Bursie wrote: On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia nt-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Could you? That is, is it only the PCH that is problematic? Trying that now. Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Weird. What if you, after a failed build, rename it to build_libjvm.pch? Doing that causes a new error: d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. cpp : fatal error C1382: the PCH file 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\build_libjvm.pch' has been rebuilt since 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\accessBarrierSupport.obj' was generated. Please rebuild this object But I think even more important is that make is not getting the error. The build just continues until interrupted. Agree, that's bad. Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation. /Magnus Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line -wsl\build\windows-x86_64-server-release\configure-support\bin\fixp ath.exe -w This starts out quite odd..? -wsl\build\...? I agree, didn't look into that part. /mnt/c/P
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
> 14 dec. 2018 kl. 23:42 skrev Erik Joelsson : > > I found the reason it's not failing make. It returns "1" and > NativeCompilation.gmk currently ignores 1 explicitly for Visual Studio. I > added that back in 2014 in https://bugs.openjdk.java.net/browse/JDK-8065576, > but I can't figure out why. Nothing mentioned in either comment or review. Sounds like it's ripe for removal then. :) I wonder what kind of issue you might have run into that caused a returned 1 to happen and yet we didn't want to consider it a failure... /Magnus > > /Erik > >> On 2018-12-14 13:59, Magnus Ihse Bursie wrote: >> >> >>> On 2018-12-14 22:15, Erik Joelsson wrote: >>> I get the same error for pch and it still continues, but this time I let it >>> run until it eventually fails for real when it can't link. Perhaps it's >>> simply cl.exe that isn't returning non zero for this error? When the linker >>> fails, make fails, so propagation doesn't seem broken. >> That does also seem really weird, considering that it claims it to be a >> "fatal error". Can you repeat the command at the command line and get the >> failure again, and then check the return value? Can you rewrite the command >> line and run it from the devenv prompt? That is, is there any indication >> that the pch file itself is messed up, or can it be used if running the >> compilation that should use it from an "ok" prompt? >> >> /Magnus >>> >>> /Erik >>> >>>> On 2018-12-14 12:55, Andrew Luo wrote: >>>> Hmm, I get the rc.exe error as well, but now it is much later down the >>>> line... Still investigating... >>>> >>>> Thanks, >>>> >>>> -Andrew >>>> >>>> -Original Message- >>>> From: Andrew Luo >>>> Sent: Friday, December 14, 2018 12:34 PM >>>> To: 'Andrew Luo' ; Magnus Ihse Bursie >>>> ; Erik Joelsson >>>> Cc: build-dev@openjdk.java.net >>>> Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for >>>> Linux) on Windows >>>> >>>> Try this updated patch with some fixes... >>>> >>>> Thanks, >>>> >>>> -Andrew >>>> >>>> -Original Message- >>>> From: build-dev On Behalf Of Andrew >>>> Luo >>>> Sent: Friday, December 14, 2018 12:01 PM >>>> To: Magnus Ihse Bursie ; Erik Joelsson >>>> >>>> Cc: build-dev@openjdk.java.net >>>> Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for >>>> Linux) on Windows >>>> >>>> I think I have a fix for it. Give me a minute (or a few hours depending >>>> on if it works). >>>> >>>> Thanks, >>>> >>>> -Andrew >>>> >>>> -Original Message- >>>> From: Magnus Ihse Bursie >>>> Sent: Friday, December 14, 2018 11:42 AM >>>> To: Erik Joelsson >>>> Cc: Andrew Luo ; >>>> build-dev@openjdk.java.net >>>> Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for >>>> Linux) on Windows >>>> >>>> >>>>> 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : >>>>> >>>>> >>>>>> On 2018-12-14 11:05, Magnus Ihse Bursie wrote: >>>>>> >>>>>> >>>>>>>> On 2018-12-14 19:41, Erik Joelsson wrote: >>>>>>>> >>>>>>>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On 2018-12-14 19:23, Erik Joelsson wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I took your patch for a spin, and configure passes, but I get the >>>>>>>>> same build error I got with my patch: >>>>>>>>> >>>>>>>>> fatal error C1083: Cannot open compiler intermediate file: >>>>>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia >>>>>>>>> nt-server\libjvm\objs\build_libjvm.pch': No such file or directory >>>>>>>>> >>>>>>>>> This is repeated for every C++ file in Hotspot. I see two issues >>>>>>>>> here. First of all, I need to figu
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
gt;0 Dir(s) 192,267,493,376 bytes free >> >> D:\>dir /x >> d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs\build_libjvm.pch >> Volume in drive D is Work >> Volume Serial Number is 4ED4-C471 >> >> Directory of d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs >> >> 2018-12-14 14:4136,634,624 BUILD_LIBJVM.pch >>1 File(s) 36,634,624 bytes >>0 Dir(s) 192,267,493,376 bytes free >> >> --- >> >> /Erik >>> >>>> Thanks, >>>> >>>> -Andrew >>>> >>>> -Original Message- >>>> From: Erik Joelsson >>>> Sent: Friday, December 14, 2018 10:42 AM >>>> To: Magnus Ihse Bursie ; Andrew Luo >>>> >>>> Cc: build-dev@openjdk.java.net >>>> Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for >>>> Linux) on Windows >>>> >>>> >>>>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote: >>>>> >>>>>> On 2018-12-14 19:23, Erik Joelsson wrote: >>>>>> Hello, >>>>>> >>>>>> I took your patch for a spin, and configure passes, but I get the >>>>>> same build error I got with my patch: >>>>>> >>>>>> fatal error C1083: Cannot open compiler intermediate file: >>>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': >>>>>> >>>>>> No such file or directory >>>>>> >>>>>> This is repeated for every C++ file in Hotspot. I see two issues >>>>>> here. First of all, I need to figure out why the compiler will not >>>>>> find the file, which is clearly there. Second, why isn't this failure >>>>>> picked up by make? Somewhere the return value of cl.exe is disappearing. >>>>> Can you build without errors if you disable PCH? >>>>> >>>>> Also, a wild guess: can it be related to file permissions? Can you >>>>> read the file properly from both WSL and Windows? >>>>> >>>> It is readable, but it could be something with case. The file is actually >>>> called BUILD_LIBJVM.pch, but that is also how it's given to the compiler >>>> command line. Here is the output from DEBUG_FIXPATH: >>>> >>>> Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >>>> >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe >>>> -w >>>> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe >>>> -showIncludes >>>> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch >>>> -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS >>>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP >>>> -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 >>>> -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows >>>> -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP >>>> -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" >>>> -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 >>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles >>>> -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share >>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share >>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows >>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 >>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 >>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc >>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled >>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include >>>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include >>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base >>>> >>>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 >>>> -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 >>>> -d2Zi+ -wd4800 -WX >>>> -I/mnt/c/PROGRA~2/MICROS~1/201
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
And now I've read up on case sensitivity in WSL. https://blogs.msdn.microsoft.com/commandline/2018/02/28/per-directory-case-sensitivity-and-wsl/ The build will certainly not work on Windows with case sensitive build directories. I managed to fix this by modifying the MakeDir macro like this: # Make directory without forking mkdir if not needed. # # If a directory with an encoded space is provided, the wildcard function # sometimes returns false answers (typically if the dir existed when the # makefile was parsed, but was deleted by a previous rule). In that case, always # call mkdir regardless of what wildcard says. # # On Windows WSL, force new dirs to be case insensitive to stay compatible with # Windows native tools used in the build. # # 1: List of directories to create MakeDir = \ $(strip \ $(eval MakeDir_dirs_to_make := $(strip $(foreach d, $1, \ $(if $(findstring ?, $d), '$(call DecodeSpace, $d)', \ $(if $(wildcard $d), , $d) \ ) \ ))) \ $(if $(MakeDir_dirs_to_make), \ $(shell $(MKDIR) -p $(MakeDir_dirs_to_make)) \ $(if $(call equals, $(OPENJDK_TARGET_OS_ENV), windows.wsl), \ $(foreach d, $(MakeDir_dirs_to_make), \ $(shell $(FSUTIL) file setCaseSensitiveInfo `$(WSLPATH) -w $d` disable > /dev/null) \ ) \ ) \ ) \ ) I also had to add FSUTIL to basics.m4 as well as add WSLPATH and FSUTIL to spec.gmk.in. Now I'm hitting the same rc.exe problem with PCH enabled as before without. /Erik On 2018-12-14 15:11, Erik Joelsson wrote: On 2018-12-14 11:33, Erik Joelsson wrote: On 2018-12-14 11:05, Andrew Luo wrote: Odd, it builds fine on my system. Did you sync down the code on Windows or WSL, and to a Windows or WSL directory? My code actually lives in Windows under /mnt/c/... Yes, otherwise it wouldn't have worked at all since Windows can't reach the WSL paths. The src was cloned in Cygwin originally. I believe there is a difference (regarding case sensitivity) depending on if you are on a Windows filesystem or a WSL filesystem. I don't think this is really about case sensitivity, but it could be a symptom. It does seem to be about being case sensitive. I extracted a failing command line and pasted into a VS env CMD window, it reproduces. I then tried to build in Cygwin, into a different output directory (this failed for other reasons later, but I got far enough that I had some object files). If I changed the compile command to use the pch-file from the Cygwin based build, the command succeeded. One notable difference between these files, the file from the Cygwin build is accessible using both upper and lower case name, while the one from WSL is not. File from WSL build: --- D:\>dir d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\BUILD_LIBJVM.pch Volume in drive D is Work Volume Serial Number is 4ED4-C471 Directory of d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs 2018-12-14 12:58 36,634,624 BUILD_LIBJVM.pch 1 File(s) 36,634,624 bytes 0 Dir(s) 192,267,493,376 bytes free D:\>dir d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch Volume in drive D is Work Volume Serial Number is 4ED4-C471 Directory of d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs File Not Found --- File from Cygwin build: --- D:\>dir /x d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs\BUILD_LIBJVM.pch Volume in drive D is Work Volume Serial Number is 4ED4-C471 Directory of d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs 2018-12-14 14:41 36,634,624 BUILD_LIBJVM.pch 1 File(s) 36,634,624 bytes 0 Dir(s) 192,267,493,376 bytes free D:\>dir /x d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs\build_libjvm.pch Volume in drive D is Work Volume Serial Number is 4ED4-C471 Directory of d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs 2018-12-14 14:41 36,634,624 BUILD_LIBJVM.pch 1 File(s) 36,634,624 bytes 0 Dir(s) 192,267,493,376 bytes free --- /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 10:42 AM To: Magnus Ihse Bursie ; Andrew Luo Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Canno
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 11:33, Erik Joelsson wrote: On 2018-12-14 11:05, Andrew Luo wrote: Odd, it builds fine on my system. Did you sync down the code on Windows or WSL, and to a Windows or WSL directory? My code actually lives in Windows under /mnt/c/... Yes, otherwise it wouldn't have worked at all since Windows can't reach the WSL paths. The src was cloned in Cygwin originally. I believe there is a difference (regarding case sensitivity) depending on if you are on a Windows filesystem or a WSL filesystem. I don't think this is really about case sensitivity, but it could be a symptom. It does seem to be about being case sensitive. I extracted a failing command line and pasted into a VS env CMD window, it reproduces. I then tried to build in Cygwin, into a different output directory (this failed for other reasons later, but I got far enough that I had some object files). If I changed the compile command to use the pch-file from the Cygwin based build, the command succeeded. One notable difference between these files, the file from the Cygwin build is accessible using both upper and lower case name, while the one from WSL is not. File from WSL build: --- D:\>dir d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\BUILD_LIBJVM.pch Volume in drive D is Work Volume Serial Number is 4ED4-C471 Directory of d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs 2018-12-14 12:58 36,634,624 BUILD_LIBJVM.pch 1 File(s) 36,634,624 bytes 0 Dir(s) 192,267,493,376 bytes free D:\>dir d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch Volume in drive D is Work Volume Serial Number is 4ED4-C471 Directory of d:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs File Not Found --- File from Cygwin build: --- D:\>dir /x d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs\BUILD_LIBJVM.pch Volume in drive D is Work Volume Serial Number is 4ED4-C471 Directory of d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs 2018-12-14 14:41 36,634,624 BUILD_LIBJVM.pch 1 File(s) 36,634,624 bytes 0 Dir(s) 192,267,493,376 bytes free D:\>dir /x d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs\build_libjvm.pch Volume in drive D is Work Volume Serial Number is 4ED4-C471 Directory of d:\erik\jdk-wsl\build\cygwin\hotspot\variant-server\libjvm\objs 2018-12-14 14:41 36,634,624 BUILD_LIBJVM.pch 1 File(s) 36,634,624 bytes 0 Dir(s) 192,267,493,376 bytes free --- /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 10:42 AM To: Magnus Ihse Bursie ; Andrew Luo Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe -w /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspo
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 14:22, Magnus Ihse Bursie wrote: Hi Erik, I missed your comments a bit down in the message. Replies inline. On 2018-12-14 19:23, Erik Joelsson wrote: In configure today, we generate the SYSROOT_CFLAGS but we do not use them in the configure script. Instead we rely on INCLUDE/LIB being set. This is of course not ideal, but having to set WSLENV=INCLUDE:LIB makes it more obvious. It would be better to append SYSROOT_CFLAGS to CFLAGS for Cygwin and msys as well, but that change is not required for WSL to work. I see. I agree that we should change this behavior for all windows envs to use SYSROOT_CFLAGS. But for now, I accept the WSLENV solution then. Replacing the path works for Cygwin, but not for WSL. At least not the way we generate the VS_PATH in this patch. The VS_PATH will not inherit the PATH from the WSL environment as base, it will get the Windows PATH that was set before WSL was launched. We could perhaps improve the extraction logic to make this work better. See below. * This is a question as much to Erik as to you: is it really correct to *always* rewrite VS_PATH to unix style? (I'm talking about lines 470..486 in toolchain_windows.m4) I think not; this sounds like it will break cygwin. I think we should to this either conditionally on us running on WSL, or we should convert it to a VS_WSL_PATH instead. Or maybe I'm just missing something, I'm starting getting a bit confused about all these paths and conversions... In configure today, we do rewrite it, but it happens implicitly in the extraction script. The current version of the patch looks like what I posted. I will try to explain. In configure today, we generate extract-vs-env.bat, which end up containing lines like this (in cygwin): C:/cygwin64/bin/bash.exe -c 'echo VS_PATH=\"$PATH\" > set-vs-env.sh (note the unbalanced '. This is baffling me, but currently works in Cygwin.) ! :-o The bat file calls back to bash to evaluate the env variable. When I tried to get this to work for WSL, I had trouble getting the quoting to work. I had also forgotten about WSLENV so $PATH would not be translated, it would hold the default bash path, and for the other variables (INCLUDE and LIB) they simply did not get values in bash. My solution was to ditch bash here and just generate lines like this instead: echo VS_PATH="%PATH%" > set-vs-env.sh This outputs the pure windows versions of the variables. For Cygwin, the old construct resulted in an implicitly converted PATH variable (though still with spaces!), but LIB and INCLUDE still pure windows. This is why we already have the conversion loops for those below. With the new construct, all variables remain pure Windows, which is arguably more consistent. So to work around now having pure windows paths in VS_PATH, I added another rewrite loop. This is a bit inefficient, but has the benefit of generating space safe paths in VS_PATH, which is a must if we are to prepend them to FIXPATH. I don't think we need to worry too much about efficiency in this case. We have a lot of inefficiencies in the path handling on Windows. The important thing is to get good and sane paths out of configure, so we can work with them easily in make. I just tried the patch on Cygwin and it isn't working. The new PATH variable we create this way does not contain /usr/bin, but instead /cygdrive/c/cygwin64/usr/bin, which doesn't actually exist. Cygwin fakes /usr/bin internally, but if it's expressed as a /cygdrive path this faking fails. The consequence is that make fails to find "gawk" later. This variable extraction will need an overhaul before we can take i this patch. /Erik
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
I found the reason it's not failing make. It returns "1" and NativeCompilation.gmk currently ignores 1 explicitly for Visual Studio. I added that back in 2014 in https://bugs.openjdk.java.net/browse/JDK-8065576, but I can't figure out why. Nothing mentioned in either comment or review. /Erik On 2018-12-14 13:59, Magnus Ihse Bursie wrote: On 2018-12-14 22:15, Erik Joelsson wrote: I get the same error for pch and it still continues, but this time I let it run until it eventually fails for real when it can't link. Perhaps it's simply cl.exe that isn't returning non zero for this error? When the linker fails, make fails, so propagation doesn't seem broken. That does also seem really weird, considering that it claims it to be a "fatal error". Can you repeat the command at the command line and get the failure again, and then check the return value? Can you rewrite the command line and run it from the devenv prompt? That is, is there any indication that the pch file itself is messed up, or can it be used if running the compilation that should use it from an "ok" prompt? /Magnus /Erik On 2018-12-14 12:55, Andrew Luo wrote: Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating... Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Friday, December 14, 2018 12:34 PM To: 'Andrew Luo' ; Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : On 2018-12-14 11:05, Magnus Ihse Bursie wrote: On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia nt-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Could you? That is, is it only the PCH that is problematic? Trying that now. Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Weird. What if you, after a failed build, rename it to build_libjvm.pch? Doing that causes a new error: d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. cpp : fatal error C1382: the PCH file 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\build_libjvm.pch' has been rebuilt since 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\accessBarrierSupport.obj' was generated. Please rebuild this object But I think even more important is that make is not getting the error. The build just continues until interrupted. Agree, that's bad. Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation. /Magnus Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line -wsl\build\windows-x86_64-server-release\configure-support\bin\fixp ath.exe -w This starts out quite odd..? -wsl\build\...? I agree, didn't look into that part. /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/ Hostx86/x64/cl.exe Also, FWIW, this seems not to have been properly case treated. Which version of the patch are you using? The last one posted by Andrew: "diff15.txt". /Erik /Magnus -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v ariant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__ST
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 22:15, Erik Joelsson wrote: I get the same error for pch and it still continues, but this time I let it run until it eventually fails for real when it can't link. Perhaps it's simply cl.exe that isn't returning non zero for this error? When the linker fails, make fails, so propagation doesn't seem broken. That does also seem really weird, considering that it claims it to be a "fatal error". Can you repeat the command at the command line and get the failure again, and then check the return value? Can you rewrite the command line and run it from the devenv prompt? That is, is there any indication that the pch file itself is messed up, or can it be used if running the compilation that should use it from an "ok" prompt? /Magnus /Erik On 2018-12-14 12:55, Andrew Luo wrote: Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating... Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Friday, December 14, 2018 12:34 PM To: 'Andrew Luo' ; Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : On 2018-12-14 11:05, Magnus Ihse Bursie wrote: On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia nt-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Could you? That is, is it only the PCH that is problematic? Trying that now. Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Weird. What if you, after a failed build, rename it to build_libjvm.pch? Doing that causes a new error: d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. cpp : fatal error C1382: the PCH file 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\build_libjvm.pch' has been rebuilt since 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\accessBarrierSupport.obj' was generated. Please rebuild this object But I think even more important is that make is not getting the error. The build just continues until interrupted. Agree, that's bad. Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation. /Magnus Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line -wsl\build\windows-x86_64-server-release\configure-support\bin\fixp ath.exe -w This starts out quite odd..? -wsl\build\...? I agree, didn't look into that part. /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/ Hostx86/x64/cl.exe Also, FWIW, this seems not to have been properly case treated. Which version of the patch are you using? The last one posted by Andrew: "diff15.txt". /Erik /Magnus -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v ariant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\""
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 21:55, Andrew Luo wrote: Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating... But you didn't get this earlier? What has changed? Do you have a local history keeping track of your changes? /Magnus Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Friday, December 14, 2018 12:34 PM To: 'Andrew Luo' ; Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : On 2018-12-14 11:05, Magnus Ihse Bursie wrote: On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia nt-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Could you? That is, is it only the PCH that is problematic? Trying that now. Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Weird. What if you, after a failed build, rename it to build_libjvm.pch? Doing that causes a new error: d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. cpp : fatal error C1382: the PCH file 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\build_libjvm.pch' has been rebuilt since 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\accessBarrierSupport.obj' was generated. Please rebuild this object But I think even more important is that make is not getting the error. The build just continues until interrupted. Agree, that's bad. Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation. /Magnus Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line -wsl\build\windows-x86_64-server-release\configure-support\bin\fixp ath.exe -w This starts out quite odd..? -wsl\build\...? I agree, didn't look into that part. /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/ Hostx86/x64/cl.exe Also, FWIW, this seems not to have been properly case treated. Which version of the patch are you using? The last one posted by Andrew: "diff15.txt". /Erik /Magnus -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v ariant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/va riant-server/gensrc/adfiles -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/va riant-server/gensrc -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include -I/mnt/d/erik/jdk-wsl/build/windows-x8
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
I get the same error for pch and it still continues, but this time I let it run until it eventually fails for real when it can't link. Perhaps it's simply cl.exe that isn't returning non zero for this error? When the linker fails, make fails, so propagation doesn't seem broken. /Erik On 2018-12-14 12:55, Andrew Luo wrote: Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating... Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Friday, December 14, 2018 12:34 PM To: 'Andrew Luo' ; Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : On 2018-12-14 11:05, Magnus Ihse Bursie wrote: On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia nt-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Could you? That is, is it only the PCH that is problematic? Trying that now. Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Weird. What if you, after a failed build, rename it to build_libjvm.pch? Doing that causes a new error: d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. cpp : fatal error C1382: the PCH file 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\build_libjvm.pch' has been rebuilt since 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s erver\libjvm\objs\accessBarrierSupport.obj' was generated. Please rebuild this object But I think even more important is that make is not getting the error. The build just continues until interrupted. Agree, that's bad. Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation. /Magnus Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line -wsl\build\windows-x86_64-server-release\configure-support\bin\fixp ath.exe -w This starts out quite odd..? -wsl\build\...? I agree, didn't look into that part. /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/ Hostx86/x64/cl.exe Also, FWIW, this seems not to have been properly case treated. Which version of the patch are you using? The last one posted by Andrew: "diff15.txt". /Erik /Magnus -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v ariant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/va riant-server/gensrc/adfiles -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/va riant-server/gensrc -I/mnt/d/erik/jdk-wsl/open/src/hots
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hmm, I get the rc.exe error as well, but now it is much later down the line... Still investigating... Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Friday, December 14, 2018 12:34 PM To: 'Andrew Luo' ; Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows > 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : > > >> On 2018-12-14 11:05, Magnus Ihse Bursie wrote: >> >> >>> On 2018-12-14 19:41, Erik Joelsson wrote: >>> >>>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote: >>>> >>>> >>>>> On 2018-12-14 19:23, Erik Joelsson wrote: >>>>> Hello, >>>>> >>>>> I took your patch for a spin, and configure passes, but I get the same >>>>> build error I got with my patch: >>>>> >>>>> fatal error C1083: Cannot open compiler intermediate file: >>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia >>>>> nt-server\libjvm\objs\build_libjvm.pch': No such file or directory >>>>> >>>>> This is repeated for every C++ file in Hotspot. I see two issues here. >>>>> First of all, I need to figure out why the compiler will not find the >>>>> file, which is clearly there. Second, why isn't this failure picked up by >>>>> make? Somewhere the return value of cl.exe is disappearing. >>>> >>>> Can you build without errors if you disable PCH? >> Could you? That is, is it only the PCH that is problematic? > Trying that now. >>>> >>>> Also, a wild guess: can it be related to file permissions? Can you read >>>> the file properly from both WSL and Windows? >>> It is readable, but it could be something with case. The file is actually >>> called BUILD_LIBJVM.pch, but that is also how it's given to the compiler >>> command line. Here is the output from DEBUG_FIXPATH: >> Weird. What if you, after a failed build, rename it to build_libjvm.pch? > > Doing that causes a new error: > > d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. > cpp : fatal error C1382: the PCH file > 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s > erver\libjvm\objs\build_libjvm.pch' has been rebuilt since > 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s > erver\libjvm\objs\accessBarrierSupport.obj' was generated. Please > rebuild this object > > But I think even more important is that make is not getting the error. The > build just continues until interrupted. Agree, that's bad. Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation. /Magnus > >>> >>> Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >>> >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixp >>> ath.exe -w >> This starts out quite odd..? -wsl\build\...? > I agree, didn't look into that part. >>> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/ >>> Hostx86/x64/cl.exe >> >> >> Also, FWIW, this seems not to have been properly case treated. Which version >> of the patch are you using? > The last one posted by Andrew: "diff15.txt". > > /Erik > >> /Magnus >>> -showIncludes >>> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v >>> ariant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp >>> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS >>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo >>> -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 >>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 >>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 >>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Try this updated patch with some fixes... Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Friday, December 14, 2018 12:01 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows > 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : > > >> On 2018-12-14 11:05, Magnus Ihse Bursie wrote: >> >> >>> On 2018-12-14 19:41, Erik Joelsson wrote: >>> >>>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote: >>>> >>>> >>>>> On 2018-12-14 19:23, Erik Joelsson wrote: >>>>> Hello, >>>>> >>>>> I took your patch for a spin, and configure passes, but I get the same >>>>> build error I got with my patch: >>>>> >>>>> fatal error C1083: Cannot open compiler intermediate file: >>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia >>>>> nt-server\libjvm\objs\build_libjvm.pch': No such file or directory >>>>> >>>>> This is repeated for every C++ file in Hotspot. I see two issues here. >>>>> First of all, I need to figure out why the compiler will not find the >>>>> file, which is clearly there. Second, why isn't this failure picked up by >>>>> make? Somewhere the return value of cl.exe is disappearing. >>>> >>>> Can you build without errors if you disable PCH? >> Could you? That is, is it only the PCH that is problematic? > Trying that now. >>>> >>>> Also, a wild guess: can it be related to file permissions? Can you read >>>> the file properly from both WSL and Windows? >>> It is readable, but it could be something with case. The file is actually >>> called BUILD_LIBJVM.pch, but that is also how it's given to the compiler >>> command line. Here is the output from DEBUG_FIXPATH: >> Weird. What if you, after a failed build, rename it to build_libjvm.pch? > > Doing that causes a new error: > > d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. > cpp : fatal error C1382: the PCH file > 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s > erver\libjvm\objs\build_libjvm.pch' has been rebuilt since > 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s > erver\libjvm\objs\accessBarrierSupport.obj' was generated. Please > rebuild this object > > But I think even more important is that make is not getting the error. The > build just continues until interrupted. Agree, that's bad. Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation. /Magnus > >>> >>> Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >>> >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixp >>> ath.exe -w >> This starts out quite odd..? -wsl\build\...? > I agree, didn't look into that part. >>> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/ >>> Hostx86/x64/cl.exe >> >> >> Also, FWIW, this seems not to have been properly case treated. Which version >> of the patch are you using? > The last one posted by Andrew: "diff15.txt". > > /Erik > >> /Magnus >>> -showIncludes >>> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v >>> ariant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp >>> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS >>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo >>> -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 >>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 >>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 >>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 >>> "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 >>> -DINCLUDE_ZGC=0 >>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/va >>> riant-server/gensrc/adfiles >>> -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share >>&g
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
I think I have a fix for it. Give me a minute (or a few hours depending on if it works). Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Friday, December 14, 2018 11:42 AM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows > 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : > > >> On 2018-12-14 11:05, Magnus Ihse Bursie wrote: >> >> >>> On 2018-12-14 19:41, Erik Joelsson wrote: >>> >>>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote: >>>> >>>> >>>>> On 2018-12-14 19:23, Erik Joelsson wrote: >>>>> Hello, >>>>> >>>>> I took your patch for a spin, and configure passes, but I get the same >>>>> build error I got with my patch: >>>>> >>>>> fatal error C1083: Cannot open compiler intermediate file: >>>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\varia >>>>> nt-server\libjvm\objs\build_libjvm.pch': No such file or directory >>>>> >>>>> This is repeated for every C++ file in Hotspot. I see two issues here. >>>>> First of all, I need to figure out why the compiler will not find the >>>>> file, which is clearly there. Second, why isn't this failure picked up by >>>>> make? Somewhere the return value of cl.exe is disappearing. >>>> >>>> Can you build without errors if you disable PCH? >> Could you? That is, is it only the PCH that is problematic? > Trying that now. >>>> >>>> Also, a wild guess: can it be related to file permissions? Can you read >>>> the file properly from both WSL and Windows? >>> It is readable, but it could be something with case. The file is actually >>> called BUILD_LIBJVM.pch, but that is also how it's given to the compiler >>> command line. Here is the output from DEBUG_FIXPATH: >> Weird. What if you, after a failed build, rename it to build_libjvm.pch? > > Doing that causes a new error: > > d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport. > cpp : fatal error C1382: the PCH file > 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s > erver\libjvm\objs\build_libjvm.pch' has been rebuilt since > 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-s > erver\libjvm\objs\accessBarrierSupport.obj' was generated. Please > rebuild this object > > But I think even more important is that make is not getting the error. The > build just continues until interrupted. Agree, that's bad. Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation. /Magnus > >>> >>> Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >>> >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixp >>> ath.exe -w >> This starts out quite odd..? -wsl\build\...? > I agree, didn't look into that part. >>> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/ >>> Hostx86/x64/cl.exe >> >> >> Also, FWIW, this seems not to have been properly case treated. Which version >> of the patch are you using? > The last one posted by Andrew: "diff15.txt". > > /Erik > >> /Magnus >>> -showIncludes >>> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/v >>> ariant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp >>> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS >>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo >>> -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 >>> -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 >>> -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 >>> -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 >>> "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 >>> -DINCLUDE_ZGC=0 >>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/va >>> riant-server/gensrc/adfiles >>> -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 >>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/va >>> riant-server/gensrc >>> -I/mnt/d/erik/jdk-ws
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
> 14 dec. 2018 kl. 20:31 skrev Erik Joelsson : > > >> On 2018-12-14 11:05, Magnus Ihse Bursie wrote: >> >> >>> On 2018-12-14 19:41, Erik Joelsson wrote: >>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote: > On 2018-12-14 19:23, Erik Joelsson wrote: > Hello, > > I took your patch for a spin, and configure passes, but I get the same > build error I got with my patch: > > fatal error C1083: Cannot open compiler intermediate file: > 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': > No such file or directory > > This is repeated for every C++ file in Hotspot. I see two issues here. > First of all, I need to figure out why the compiler will not find the > file, which is clearly there. Second, why isn't this failure picked up by > make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? >> Could you? That is, is it only the PCH that is problematic? > Trying that now. Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? >>> It is readable, but it could be something with case. The file is actually >>> called BUILD_LIBJVM.pch, but that is also how it's given to the compiler >>> command line. Here is the output from DEBUG_FIXPATH: >> Weird. What if you, after a failed build, rename it to build_libjvm.pch? > > Doing that causes a new error: > > d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport.cpp : > fatal error C1382: the PCH file > 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch' > has been rebuilt since > 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\accessBarrierSupport.obj' > was generated. Please rebuild this object > > But I think even more important is that make is not getting the error. The > build just continues until interrupted. Agree, that's bad. Does fixpath_debug print exit code? If so, what does it say? If not, we should add that instrumentation. /Magnus > >>> >>> Compiling ad_x86_expand.cpp (for jvm.dll) >>> fixpath input line >>> >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe >>> -w >> This starts out quite odd..? -wsl\build\...? > I agree, didn't look into that part. >>> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe >> >> >> Also, FWIW, this seems not to have been properly case treated. Which version >> of the patch are you using? > The last one posted by Andrew: "diff15.txt". > > /Erik > >> /Magnus >>> -showIncludes >>> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch >>> -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS >>> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP >>> -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 >>> -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows >>> -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP >>> -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 >>> -DCOMPILER2 -DINCLUDE_ZGC=0 >>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles >>> -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 >>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include >>> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include >>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base >>> >>> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 >>> -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ >>> -wd4800 -WX >>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include >>> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include >>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt >>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared >>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um >>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt >>> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- >>> "-DTHIS_FILE=\"\"" -c >>> -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj >>> >>>
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Is link.exe locating and executing rc.exe? How does it find it? /Magnus > 14 dec. 2018 kl. 20:37 skrev Erik Joelsson : > > >> On 2018-12-14 11:31, Erik Joelsson wrote: >> >>> On 2018-12-14 11:05, Magnus Ihse Bursie wrote: >>> Can you build without errors if you disable PCH? >>> Could you? That is, is it only the PCH that is problematic? >> Trying that now. > That got much farther, but ultimately failed on: > > LINK : fatal error LNK1158: cannot run 'rc.exe' > > The definition of RC does look correct in spec.gmk though. > > /Erik >
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
/Magnus > 14 dec. 2018 kl. 20:33 skrev Erik Joelsson : > >> On 2018-12-14 11:05, Andrew Luo wrote: >> Odd, it builds fine on my system. Did you sync down the code on Windows or >> WSL, and to a Windows or WSL directory? My code actually lives in Windows >> under /mnt/c/... > Yes, otherwise it wouldn't have worked at all since Windows can't reach the > WSL paths. The src was cloned in Cygwin originally. Any difference if you clone in wsl directly? (Also create the directory that you're cloning in from wsl...) /Magnus >> I believe there is a difference (regarding case sensitivity) depending on if >> you are on a Windows filesystem or a WSL filesystem. > > I don't think this is really about case sensitivity, but it could be a > symptom. > > /Erik > >> Thanks, >> >> -Andrew >> >> -Original Message- >> From: Erik Joelsson >> Sent: Friday, December 14, 2018 10:42 AM >> To: Magnus Ihse Bursie ; Andrew Luo >> >> Cc: build-dev@openjdk.java.net >> Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for >> Linux) on Windows >> >> >>> On 2018-12-14 10:28, Magnus Ihse Bursie wrote: >>> >>>> On 2018-12-14 19:23, Erik Joelsson wrote: >>>> Hello, >>>> >>>> I took your patch for a spin, and configure passes, but I get the >>>> same build error I got with my patch: >>>> >>>> fatal error C1083: Cannot open compiler intermediate file: >>>> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': >>>> No such file or directory >>>> >>>> This is repeated for every C++ file in Hotspot. I see two issues >>>> here. First of all, I need to figure out why the compiler will not >>>> find the file, which is clearly there. Second, why isn't this failure >>>> picked up by make? Somewhere the return value of cl.exe is disappearing. >>> Can you build without errors if you disable PCH? >>> >>> Also, a wild guess: can it be related to file permissions? Can you >>> read the file properly from both WSL and Windows? >> It is readable, but it could be something with case. The file is actually >> called BUILD_LIBJVM.pch, but that is also how it's given to the compiler >> command line. Here is the output from DEBUG_FIXPATH: >> >> Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >> >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe >> -w >> /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe >> -showIncludes >> -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch >> -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS >> -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP >> -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 >> -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows >> -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP >> -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 >> -DCOMPILER2 -DINCLUDE_ZGC=0 >> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles >> -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share >> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share >> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows >> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 >> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 >> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc >> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled >> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include >> -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include >> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base >> >> -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 >> -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ >> -wd4800 -WX >> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include >> -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include >> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt >> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared >> -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um >> -I/mnt/c/PROGRA~2/W
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 11:31, Erik Joelsson wrote: On 2018-12-14 11:05, Magnus Ihse Bursie wrote: Can you build without errors if you disable PCH? Could you? That is, is it only the PCH that is problematic? Trying that now. That got much farther, but ultimately failed on: LINK : fatal error LNK1158: cannot run 'rc.exe' The definition of RC does look correct in spec.gmk though. /Erik
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 11:05, Andrew Luo wrote: Odd, it builds fine on my system. Did you sync down the code on Windows or WSL, and to a Windows or WSL directory? My code actually lives in Windows under /mnt/c/... Yes, otherwise it wouldn't have worked at all since Windows can't reach the WSL paths. The src was cloned in Cygwin originally. I believe there is a difference (regarding case sensitivity) depending on if you are on a Windows filesystem or a WSL filesystem. I don't think this is really about case sensitivity, but it could be a symptom. /Erik Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 10:42 AM To: Magnus Ihse Bursie ; Andrew Luo Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe -w /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ -wd4800 -WX -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- "-DTHIS_FILE=\"\"" -c -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles/ad_x86_expand.cpp< fixpath using wsl mode, with path list: fixpath converted line >c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DC
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 11:05, Magnus Ihse Bursie wrote: On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Could you? That is, is it only the PCH that is problematic? Trying that now. Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Weird. What if you, after a failed build, rename it to build_libjvm.pch? Doing that causes a new error: d:\erik\jdk-wsl\open\src\hotspot\share\gc\shared\accessBarrierSupport.cpp : fatal error C1382: the PCH file 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch' has been rebuilt since 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\accessBarrierSupport.obj' was generated. Please rebuild this object But I think even more important is that make is not getting the error. The build just continues until interrupted. Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe -w This starts out quite odd..? -wsl\build\...? I agree, didn't look into that part. /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe Also, FWIW, this seems not to have been properly case treated. Which version of the patch are you using? The last one posted by Andrew: "diff15.txt". /Erik /Magnus -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ -wd4800 -WX -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- "-DTHIS_FILE=\"\"" -c -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles/ad_x86_expand.cpp< fixpath using wsl mode, with path list: fixpath converted line >c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 20:05, Andrew Luo wrote: Odd, it builds fine on my system. Did you sync down the code on Windows or WSL, and to a Windows or WSL directory? My code actually lives in Windows under /mnt/c/... And the build directory too, I presume..? It looked on Erik's path that it was Windows, d:\erik\jdk-wsl\build... Can the Windows tools (e.g. cl.exe) even reach files in a WSL directory? /Magnus I believe there is a difference (regarding case sensitivity) depending on if you are on a Windows filesystem or a WSL filesystem. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 10:42 AM To: Magnus Ihse Bursie ; Andrew Luo Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe -w /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ -wd4800 -WX -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- "-DTHIS_FILE=\"\"" -c -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles/ad_x86_expand.cpp< fixpath using wsl mode, with path list: fixpath converted line >c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOM
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 19:41, Erik Joelsson wrote: On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Could you? That is, is it only the PCH that is problematic? Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Weird. What if you, after a failed build, rename it to build_libjvm.pch? Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe -w This starts out quite odd..? -wsl\build\...? /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe Also, FWIW, this seems not to have been properly case treated. Which version of the patch are you using? /Magnus -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ -wd4800 -WX -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- "-DTHIS_FILE=\"\"" -c -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles/ad_x86_expand.cpp< fixpath using wsl mode, with path list: fixpath converted line >c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -Id:/erik/jdk-wsl/closed/src/hotspot/share -Id:/erik/jdk-wsl/open/src/hotspot/share -Id:/erik/jdk-wsl/open/src/hotspot/os/windows -Id:/erik/jdk-wsl/open/src/hotspot/cpu/x86 -Id:/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc -Id:/erik/jdk-wsl/open/src/hotspot/share/precompiled
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Odd, it builds fine on my system. Did you sync down the code on Windows or WSL, and to a Windows or WSL directory? My code actually lives in Windows under /mnt/c/... I believe there is a difference (regarding case sensitivity) depending on if you are on a Windows filesystem or a WSL filesystem. Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Friday, December 14, 2018 10:42 AM To: Magnus Ihse Bursie ; Andrew Luo Cc: build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-14 10:28, Magnus Ihse Bursie wrote: > > > On 2018-12-14 19:23, Erik Joelsson wrote: >> Hello, >> >> I took your patch for a spin, and configure passes, but I get the >> same build error I got with my patch: >> >> fatal error C1083: Cannot open compiler intermediate file: >> 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': >> >> No such file or directory >> >> This is repeated for every C++ file in Hotspot. I see two issues >> here. First of all, I need to figure out why the compiler will not >> find the file, which is clearly there. Second, why isn't this failure >> picked up by make? Somewhere the return value of cl.exe is disappearing. > > Can you build without errors if you disable PCH? > > Also, a wild guess: can it be related to file permissions? Can you > read the file properly from both WSL and Windows? > It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe -w /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ -wd4800 -WX -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- "-DTHIS_FILE=\"\"" -c -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles/ad_x86_expand.cpp< fixpath using wsl mode, with path list: fixpath converted line >c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe > -showIncludes >-Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch > -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS >-D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP >-D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 >-DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows >-DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP >-DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 >-DCOMPILER2 -DINCLUDE_
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 10:28, Magnus Ihse Bursie wrote: On 2018-12-14 19:23, Erik Joelsson wrote: Hello, I took your patch for a spin, and configure passes, but I get the same build error I got with my patch: fatal error C1083: Cannot open compiler intermediate file: 'd:\erik\jdk-wsl\build\windows-x86_64-server-release\hotspot\variant-server\libjvm\objs\build_libjvm.pch': No such file or directory This is repeated for every C++ file in Hotspot. I see two issues here. First of all, I need to figure out why the compiler will not find the file, which is clearly there. Second, why isn't this failure picked up by make? Somewhere the return value of cl.exe is disappearing. Can you build without errors if you disable PCH? Also, a wild guess: can it be related to file permissions? Can you read the file properly from both WSL and Windows? It is readable, but it could be something with case. The file is actually called BUILD_LIBJVM.pch, but that is also how it's given to the compiler command line. Here is the output from DEBUG_FIXPATH: Compiling ad_x86_expand.cpp (for jvm.dll) fixpath input line >-wsl\build\windows-x86_64-server-release\configure-support\bin\fixpath.exe -w /mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fp/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -I/mnt/d/erik/jdk-wsl/closed/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows -I/mnt/d/erik/jdk-wsl/open/src/hotspot/cpu/x86 -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/precompiled -I/mnt/d/erik/jdk-wsl/open/src/hotspot/share/include -I/mnt/d/erik/jdk-wsl/open/src/hotspot/os/windows/include -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base -I/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 -I/mnt/d/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ -wd4800 -WX -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/atlmfc/include -I/mnt/c/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/include -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/ucrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/shared -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/um -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/winrt -I/mnt/c/PROGRA~2/WI3CF2~1/10/Include/100177~1.0/cppwinrt -O2 -Oy- "-DTHIS_FILE=\"\"" -c -Fo/mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/ad_x86_expand.obj /mnt/d/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles/ad_x86_expand.cpp< fixpath using wsl mode, with path list: fixpath converted line >c:/PROGRA~2/MICROS~1/2017/PROFES~1/VC/Tools/MSVC/1416~1.270/bin/Hostx86/x64/cl.exe -showIncludes -Fpd:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objs/BUILD_LIBJVM.pch -Yuprecompiled.hpp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -nologo -MD -MP -D_WINDOWS -DWIN32 -D_JNI_IMPLEMENTATION_ -W3 -DVM_LITTLE_ENDIAN -D_LP64=1 -DPRODUCT -DTARGET_ARCH_x86 -DINCLUDE_SUFFIX_OS=_windows -DINCLUDE_SUFFIX_CPU=_x86 -DINCLUDE_SUFFIX_COMPILER=_visCPP -DTARGET_COMPILER_visCPP -DAMD64 "-DHOTSPOT_LIB_ARCH=\"amd64\"" -DCOMPILER1 -DCOMPILER2 -DINCLUDE_ZGC=0 -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc/adfiles -Id:/erik/jdk-wsl/closed/src/hotspot/share -Id:/erik/jdk-wsl/open/src/hotspot/share -Id:/erik/jdk-wsl/open/src/hotspot/os/windows -Id:/erik/jdk-wsl/open/src/hotspot/cpu/x86 -Id:/erik/jdk-wsl/open/src/hotspot/os_cpu/windows_x86 -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/hotspot/variant-server/gensrc -Id:/erik/jdk-wsl/open/src/hotspot/share/precompiled -Id:/erik/jdk-wsl/open/src/hotspot/share/include -Id:/erik/jdk-wsl/open/src/hotspot/os/windows/include -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base -Id:/erik/jdk-wsl/build/windows-x86_64-server-release/support/modules_include/java.base/win32 -Id:/erik/jdk-wsl/open/src/java.base/share/native/libjimage -Z7 -d2Zi+ -wd4800 -WX
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
iables (INCLUDE and LIB) they simply did not get values in bash. My solution was to ditch bash here and just generate lines like this instead: echo VS_PATH="%PATH%" > set-vs-env.sh This outputs the pure windows versions of the variables. For Cygwin, the old construct resulted in an implicitly converted PATH variable (though still with spaces!), but LIB and INCLUDE still pure windows. This is why we already have the conversion loops for those below. With the new construct, all variables remain pure Windows, which is arguably more consistent. So to work around now having pure windows paths in VS_PATH, I added another rewrite loop. This is a bit inefficient, but has the benefit of generating space safe paths in VS_PATH, which is a must if we are to prepend them to FIXPATH. Maybe we can revert some of this and get back to something closer to the current behavior using WSLENV in extract-vs-env.bat, but converting PATH between Windows and WSL using WSLENV is tricky as we have seen. The WSL unix specific paths (/bin:/usr/bin etc) simply do not translate to windows so will get lost when going back and forth. For this reason, I do not think we can get around having to append the VS_PATH to PATH instead of replacing. I certainly feel like there are possible improvements to the whole scheme. * In toolchain.m4, line 392, looks like the ; should be before $path, not after..? Also, this logic should be in toolchain_windows.m4, and not "pollute" the generic toolchain.m4. I would recommend the macros BASIC_PREPEND_TO_PATH and BASIC_APPEND_TO_PATH which I created specifically for this situation. * The two fixes filtering out UtilTranslatePathList should not be needed anymore. * I noted that the windowsShortName.bat file is missing from your patch. Perhaps you forgot a "hg add"? * In fixpath.c, it looks like you're trying to separate the original path and the fixpath_path with a : (colon), not ; (semicolon). Since this is a Windows path, it should be the latter. * In Images.gmk, I'd like to see a FIXPATH prefix rather than the EXE_SUFFIX. * I'm not entirely certain about the addition of literal ".exe" to all Windows toolchain binaries. It's easier to read than $(EXE_SUFFIX), but the latter makes it abuntantly clear that it's needed. I feel there's a certain risk someone editing those files in the future think, "hey, this .exe is just ugly and is not needed, let's remove it". But then again, I'm not sure about this, and I'd like to hear Erik's opinion on it. I added those in my patch, and did it because we had some tools that already defined .exe as suffix. I just figured since we were in a microsoft specific block, .exe looked better. We could standardize on EXE_SUFFIX. We seem to have a duplication though as EXECUTABLE_SUFFIX is also present in some locations. /Erik * I'm curious of the change in filename case in "VC/Auxiliary/Build". Did you encounter any issues regarding filename case, or is it just a matter of aesthetics? * Did you get around to looking if there's any relevant environment version information to add to the configure log? I googled around a bit, it seems that WSL releases are tied 1-to-1 with Windows releases, so running "ver.exe" might do the trick. (See https://github.com/Microsoft/WSL/issues/2125). /Magnus Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Thursday, December 13, 2018 11:07 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Didn't see this message before I sent mine out. How about an environment variable instead? I don't want to make too much changes to the argument parsing logic, etc. of fixpath, instead in -w mode it could read an environment variable, perhaps FIXPATH_PATH and set PATH to that? Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Thursday, December 13, 2018 10:36 PM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Maybe we can get fixpath to help here? It could take an extra argument with -w, the additional directories to add to PATH before executing the target command? /Magnus 13 dec. 2018 kl. 21:36 skrev Erik Joelsson : On 2018-12-13 11:44, Andrew Luo wrote: Oh, also, using WSLPATH to set PATH/l causes a LOT of extra output, namely, every time a Win32 executable is run this gets printed: <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/sbin <3>init: (21845) ERROR: UtilTranslatePathList:
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
ersions of the variables. For Cygwin, the old construct resulted in an implicitly converted PATH variable (though still with spaces!), but LIB and INCLUDE still pure windows. This is why we already have the conversion loops for those below. With the new construct, all variables remain pure Windows, which is arguably more consistent. So to work around now having pure windows paths in VS_PATH, I added another rewrite loop. This is a bit inefficient, but has the benefit of generating space safe paths in VS_PATH, which is a must if we are to prepend them to FIXPATH. Maybe we can revert some of this and get back to something closer to the current behavior using WSLENV in extract-vs-env.bat, but converting PATH between Windows and WSL using WSLENV is tricky as we have seen. The WSL unix specific paths (/bin:/usr/bin etc) simply do not translate to windows so will get lost when going back and forth. For this reason, I do not think we can get around having to append the VS_PATH to PATH instead of replacing. I certainly feel like there are possible improvements to the whole scheme. * In toolchain.m4, line 392, looks like the ; should be before $path, not after..? Also, this logic should be in toolchain_windows.m4, and not "pollute" the generic toolchain.m4. I would recommend the macros BASIC_PREPEND_TO_PATH and BASIC_APPEND_TO_PATH which I created specifically for this situation. * The two fixes filtering out UtilTranslatePathList should not be needed anymore. * I noted that the windowsShortName.bat file is missing from your patch. Perhaps you forgot a "hg add"? * In fixpath.c, it looks like you're trying to separate the original path and the fixpath_path with a : (colon), not ; (semicolon). Since this is a Windows path, it should be the latter. * In Images.gmk, I'd like to see a FIXPATH prefix rather than the EXE_SUFFIX. * I'm not entirely certain about the addition of literal ".exe" to all Windows toolchain binaries. It's easier to read than $(EXE_SUFFIX), but the latter makes it abuntantly clear that it's needed. I feel there's a certain risk someone editing those files in the future think, "hey, this .exe is just ugly and is not needed, let's remove it". But then again, I'm not sure about this, and I'd like to hear Erik's opinion on it. I added those in my patch, and did it because we had some tools that already defined .exe as suffix. I just figured since we were in a microsoft specific block, .exe looked better. We could standardize on EXE_SUFFIX. We seem to have a duplication though as EXECUTABLE_SUFFIX is also present in some locations. /Erik * I'm curious of the change in filename case in "VC/Auxiliary/Build". Did you encounter any issues regarding filename case, or is it just a matter of aesthetics? * Did you get around to looking if there's any relevant environment version information to add to the configure log? I googled around a bit, it seems that WSL releases are tied 1-to-1 with Windows releases, so running "ver.exe" might do the trick. (See https://github.com/Microsoft/WSL/issues/2125). /Magnus Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Thursday, December 13, 2018 11:07 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Didn't see this message before I sent mine out. How about an environment variable instead? I don't want to make too much changes to the argument parsing logic, etc. of fixpath, instead in -w mode it could read an environment variable, perhaps FIXPATH_PATH and set PATH to that? Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Thursday, December 13, 2018 10:36 PM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Maybe we can get fixpath to help here? It could take an extra argument with -w, the additional directories to add to PATH before executing the target command? /Magnus 13 dec. 2018 kl. 21:36 skrev Erik Joelsson : On 2018-12-13 11:44, Andrew Luo wrote: Oh, also, using WSLPATH to set PATH/l causes a LOT of extra output, namely, every time a Win32 executable is run this gets printed: <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /bin <3>init: (21845) ERROR: Uti
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-14 09:58, Andrew Luo wrote: Anyways, got it working with an environment variable, patch is attached. To use: 1. Open WSL 2. Run configure, pass in boot JDK 3. Run make (no extra error messages here). This is starting to look really really nice! We're getting WSL support, and some extra improvement for other environments as well! :-) I'm glad the basic idea seem to work out! Now the usage description is down to exactly what we want! :) I still think we can polish the path mangling a bit more, though. Or possibly, I'm just lost and confused about what's happening right now. :-) I'll try to write down what I think looks a bit odd: * In spec.gmk.in; why do you change PATH for WSL? I thought we could let PATH stay the way it is, and just set FIXPATH_PATH? (And tell WSL to process FIXPATH_PATH). * Ideally, I was hoping we could be able to change the $(FIXPATH) expression to be something like: FIXPATH=WSLENV:=FIXPATH_PATH FIXPATH_PATH=$(FIXPATH_PATH) $TOPDIR/.../fixpath.exe That would mean that we could finally get 100% rid of environment variables, and have a fully copy-and-pastable command line, even for Windows tools. (Something we don't have even for cygwin today). In fact, if we can get this to work, I'd like to switch to using such a system (but without the WSLENV) for cygwin/msys as well. * I also note that you add INCLUDE/LIB to WSLENV in places. The code for this in basics_windows.m4 (line 546) is completele beyond me. What are you trying to achieve here? Also in toolchain.m4; why is not setting PATH to VS_PATH enough? The INCLUDE and LIB variables is not/should not be needed, since we convert these to SYSROOT flags and pass them alongside the compiler. Possibly there is some "bootstrapping" problems in toolchain.m4 which makes us at times run the compiler without SYSROOT flags. Maybe that's what you're running into? But in any case you should not append VS_PATH to PATH, since VS_PATH contains the entire old PATH + what was added by the msdevenv script. (It would be nice if we could filter out just what was added, but that might be tricky.) So replacing PATH with VS_PATH should be just fine, both in the old code and in the WSL case. * This is a question as much to Erik as to you: is it really correct to *always* rewrite VS_PATH to unix style? (I'm talking about lines 470..486 in toolchain_windows.m4) I think not; this sounds like it will break cygwin. I think we should to this either conditionally on us running on WSL, or we should convert it to a VS_WSL_PATH instead. Or maybe I'm just missing something, I'm starting getting a bit confused about all these paths and conversions... * In toolchain.m4, line 392, looks like the ; should be before $path, not after..? Also, this logic should be in toolchain_windows.m4, and not "pollute" the generic toolchain.m4. * The two fixes filtering out UtilTranslatePathList should not be needed anymore. * I noted that the windowsShortName.bat file is missing from your patch. Perhaps you forgot a "hg add"? * In fixpath.c, it looks like you're trying to separate the original path and the fixpath_path with a : (colon), not ; (semicolon). Since this is a Windows path, it should be the latter. * In Images.gmk, I'd like to see a FIXPATH prefix rather than the EXE_SUFFIX. * I'm not entirely certain about the addition of literal ".exe" to all Windows toolchain binaries. It's easier to read than $(EXE_SUFFIX), but the latter makes it abuntantly clear that it's needed. I feel there's a certain risk someone editing those files in the future think, "hey, this .exe is just ugly and is not needed, let's remove it". But then again, I'm not sure about this, and I'd like to hear Erik's opinion on it. * I'm curious of the change in filename case in "VC/Auxiliary/Build". Did you encounter any issues regarding filename case, or is it just a matter of aesthetics? * Did you get around to looking if there's any relevant environment version information to add to the configure log? I googled around a bit, it seems that WSL releases are tied 1-to-1 with Windows releases, so running "ver.exe" might do the trick. (See https://github.com/Microsoft/WSL/issues/2125). /Magnus Thanks, -Andrew -Original Message- From: build-dev On Behalf Of Andrew Luo Sent: Thursday, December 13, 2018 11:07 PM To: Magnus Ihse Bursie ; Erik Joelsson Cc: build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Didn't see this message before I sent mine out. How about an environment variable instead? I don't want to make too much changes to the argument parsing logic, etc. of fixpath, instead in -w mode it could read an environment variable, perhaps FIXPATH_PATH and set PATH to that? Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Thursday, De
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Didn't see this message before I sent mine out. How about an environment variable instead? I don't want to make too much changes to the argument parsing logic, etc. of fixpath, instead in -w mode it could read an environment variable, perhaps FIXPATH_PATH and set PATH to that? Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Thursday, December 13, 2018 10:36 PM To: Erik Joelsson Cc: Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Maybe we can get fixpath to help here? It could take an extra argument with -w, the additional directories to add to PATH before executing the target command? /Magnus > 13 dec. 2018 kl. 21:36 skrev Erik Joelsson : > >> On 2018-12-13 11:44, Andrew Luo wrote: >> Oh, also, using WSLPATH to set PATH/l causes a LOT of extra output, namely, >> every time a Win32 executable is run this gets printed: >> >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /usr/local/sbin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /usr/local/bin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /usr/sbin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /usr/bin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /sbin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /bin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /usr/games >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /usr/local/games >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /snap/bin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /usr/local/sbin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to >> translate /usr/local/sbin >> >> Don't know if there's any way to tell WSL to suppress it. > > Hm, that becomes tricky. We need to export a PATH to Windows processes > (Visual Studio processes really) that includes certain VS dirs so that they > can load the dlls they need. It would be more convenient if that windows path > could be stored in a different environment variable, but it doesn't seem like > WSLENV can map between different names. Another approach could be to prefix > the affected commands (CC etc) with something like "WSLENV=PATH/l PATH=...", > with a filtered down version of the VS_PATH. That would also have the added > benefit of making the commands we put in *.cmdline files be directly > executable in a pure shell. Today those commands won't work since they rely > on an exported PATH, even in Cygwin. > > /Erik > >> Thanks, >> >> -Andrew >> >> -Original Message- >> From: Andrew Luo >> Sent: Thursday, December 13, 2018 11:43 AM >> To: Erik Joelsson ; Magnus Ihse Bursie >> ; build-dev@openjdk.java.net >> Subject: RE: [PATCH] Support for building using WSL (Windows >> Subsystem for Linux) on Windows >> >> Hi Erik, >> >> Thanks for helping out on this. I made some changes to your patch and can >> get it working on my system. It's still building but it seems to be >> working. Will update this thread once it's finished building... >> >> Thanks, >> >> -Andrew >> >> -Original Message- >> From: Erik Joelsson >> Sent: Thursday, December 13, 2018 10:36 AM >> To: Magnus Ihse Bursie ; Andrew Luo >> ; build-dev@openjdk.java.net >> Subject: Re: [PATCH] Support for building using WSL (Windows >> Subsystem for Linux) on Windows >> >> Hello, >> >> It's nice to see this progressing! >> >> I just wanted to let you know I took your patch from yesterday and started >> experimenting too. I managed to get configure to automatically find the >> Visual Studio installation as Magnus described, when running in a pure WSL >> shell without VS env. I still have some issues with the build though so the >> patch is not fully working yet. One thing I'm still experimenting with is >> the method of extraction of the VS variables. I think this can be improved >> more. In the old implementation we are relying on automatic conversion and >> sharing of some variables between Windows and the unix emulation (which >> Cygwin does for PATH and msys magically tries to do for all sorts of stuff). >> These need explicit declaration
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
I just tried your suggestion. $CC is passed as a parameter to fixpath sometimes, so it will require a decent bit of work to get working. Also, when executing commands on bash in the format "$CC ..." $CC has to be an actual command - it can't have extra environment variables set at the beginning. If you want to do that, you have to use eval, so you can set WSL_CC="PATH=... $OLD_CC" and then CC="eval $WSL_CC", but that fixpath broke... That said, the build (make + make images) worked anyways despite the extra error messages. So we have two options basically: 1. Set WSLPATH and have a bunch of extra output for failed path translation 2. Force users to start WSL from a Visual Studio environment (start VC++ Tools CMD, set WSLENV, run wsl) Personally I prefer (2) since (1) causes a lot of extra debug output. If you do prefer (1) however, I don't have any objections, and my previous patch (diff13.txt) does that. (1) does have some advantages since later on perhaps we can ask Microsoft's WSL team to add an environment variable to disable those warnings... Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Thursday, December 13, 2018 12:37 PM To: Andrew Luo ; Magnus Ihse Bursie ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-13 11:44, Andrew Luo wrote: > Oh, also, using WSLPATH to set PATH/l causes a LOT of extra output, namely, > every time a Win32 executable is run this gets printed: > > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /usr/local/sbin > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /usr/local/bin > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /usr/sbin > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /usr/bin > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /sbin > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /bin > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /usr/games > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /usr/local/games > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /snap/bin > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /usr/local/sbin > <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to > translate /usr/local/sbin > > Don't know if there's any way to tell WSL to suppress it. Hm, that becomes tricky. We need to export a PATH to Windows processes (Visual Studio processes really) that includes certain VS dirs so that they can load the dlls they need. It would be more convenient if that windows path could be stored in a different environment variable, but it doesn't seem like WSLENV can map between different names. Another approach could be to prefix the affected commands (CC etc) with something like "WSLENV=PATH/l PATH=...", with a filtered down version of the VS_PATH. That would also have the added benefit of making the commands we put in *.cmdline files be directly executable in a pure shell. Today those commands won't work since they rely on an exported PATH, even in Cygwin. /Erik > Thanks, > > -Andrew > > -Original Message----- > From: Andrew Luo > Sent: Thursday, December 13, 2018 11:43 AM > To: Erik Joelsson ; Magnus Ihse Bursie > ; build-dev@openjdk.java.net > Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem > for Linux) on Windows > > Hi Erik, > > Thanks for helping out on this. I made some changes to your patch and can > get it working on my system. It's still building but it seems to be working. > Will update this thread once it's finished building... > > Thanks, > > -Andrew > > -Original Message- > From: Erik Joelsson > Sent: Thursday, December 13, 2018 10:36 AM > To: Magnus Ihse Bursie ; Andrew Luo > ; build-dev@openjdk.java.net > Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem > for Linux) on Windows > > Hello, > > It's nice to see this progressing! > > I just wanted to let you know I took your patch from yesterday and started > experimenting too. I managed to get configure to automatically find the > Visual Studio installation as Magnus described, when running in a pure WSL > shell without VS env. I still have some issues with the build though so the > patch is not fully working yet. One thing I'm still experimenting with is the > method of extraction of the VS variables. I think this can be improved more. > In the old implementation we are relying on a
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Maybe we can get fixpath to help here? It could take an extra argument with -w, the additional directories to add to PATH before executing the target command? /Magnus > 13 dec. 2018 kl. 21:36 skrev Erik Joelsson : > >> On 2018-12-13 11:44, Andrew Luo wrote: >> Oh, also, using WSLPATH to set PATH/l causes a LOT of extra output, namely, >> every time a Win32 executable is run this gets printed: >> >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate >> /usr/local/sbin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate >> /usr/local/bin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate >> /usr/sbin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate >> /usr/bin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /sbin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /bin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate >> /usr/games >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate >> /usr/local/games >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate >> /snap/bin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate >> /usr/local/sbin >> <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate >> /usr/local/sbin >> >> Don't know if there's any way to tell WSL to suppress it. > > Hm, that becomes tricky. We need to export a PATH to Windows processes > (Visual Studio processes really) that includes certain VS dirs so that they > can load the dlls they need. It would be more convenient if that windows path > could be stored in a different environment variable, but it doesn't seem like > WSLENV can map between different names. Another approach could be to prefix > the affected commands (CC etc) with something like "WSLENV=PATH/l PATH=...", > with a filtered down version of the VS_PATH. That would also have the added > benefit of making the commands we put in *.cmdline files be directly > executable in a pure shell. Today those commands won't work since they rely > on an exported PATH, even in Cygwin. > > /Erik > >> Thanks, >> >> -Andrew >> >> -Original Message- >> From: Andrew Luo >> Sent: Thursday, December 13, 2018 11:43 AM >> To: Erik Joelsson ; Magnus Ihse Bursie >> ; build-dev@openjdk.java.net >> Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for >> Linux) on Windows >> >> Hi Erik, >> >> Thanks for helping out on this. I made some changes to your patch and can >> get it working on my system. It's still building but it seems to be >> working. Will update this thread once it's finished building... >> >> Thanks, >> >> -Andrew >> >> -Original Message- >> From: Erik Joelsson >> Sent: Thursday, December 13, 2018 10:36 AM >> To: Magnus Ihse Bursie ; Andrew Luo >> ; build-dev@openjdk.java.net >> Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for >> Linux) on Windows >> >> Hello, >> >> It's nice to see this progressing! >> >> I just wanted to let you know I took your patch from yesterday and started >> experimenting too. I managed to get configure to automatically find the >> Visual Studio installation as Magnus described, when running in a pure WSL >> shell without VS env. I still have some issues with the build though so the >> patch is not fully working yet. One thing I'm still experimenting with is >> the method of extraction of the VS variables. I think this can be improved >> more. In the old implementation we are relying on automatic conversion and >> sharing of some variables between Windows and the unix emulation (which >> Cygwin does for PATH and msys magically tries to do for all sorts of stuff). >> These need explicit declaration using WSLENV in WSL, but that also gives us >> a lot of control over it. >> >> I have limited time to spend on this, so uploading the patch here for >> reference. Perhaps there is something there that could inspire you at least. >> I may get more time to revisit this next week or so. >> >> http://cr.openjdk.java.net/~erikj/windows-wsl/webrev.01/index.html >> >> /Erik >> >>> On 2018-12-13 03:12, Magnus Ihse Bursie wrote: >>>> On 2018-12-13 08:48, Andrew Luo wrote: >>>> Hi, >
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-13 11:44, Andrew Luo wrote: Oh, also, using WSLPATH to set PATH/l causes a LOT of extra output, namely, every time a Win32 executable is run this gets printed: <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/games <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/games <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /snap/bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/sbin Don't know if there's any way to tell WSL to suppress it. Hm, that becomes tricky. We need to export a PATH to Windows processes (Visual Studio processes really) that includes certain VS dirs so that they can load the dlls they need. It would be more convenient if that windows path could be stored in a different environment variable, but it doesn't seem like WSLENV can map between different names. Another approach could be to prefix the affected commands (CC etc) with something like "WSLENV=PATH/l PATH=...", with a filtered down version of the VS_PATH. That would also have the added benefit of making the commands we put in *.cmdline files be directly executable in a pure shell. Today those commands won't work since they rely on an exported PATH, even in Cygwin. /Erik Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Thursday, December 13, 2018 11:43 AM To: Erik Joelsson ; Magnus Ihse Bursie ; build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hi Erik, Thanks for helping out on this. I made some changes to your patch and can get it working on my system. It's still building but it seems to be working. Will update this thread once it's finished building... Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Thursday, December 13, 2018 10:36 AM To: Magnus Ihse Bursie ; Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello, It's nice to see this progressing! I just wanted to let you know I took your patch from yesterday and started experimenting too. I managed to get configure to automatically find the Visual Studio installation as Magnus described, when running in a pure WSL shell without VS env. I still have some issues with the build though so the patch is not fully working yet. One thing I'm still experimenting with is the method of extraction of the VS variables. I think this can be improved more. In the old implementation we are relying on automatic conversion and sharing of some variables between Windows and the unix emulation (which Cygwin does for PATH and msys magically tries to do for all sorts of stuff). These need explicit declaration using WSLENV in WSL, but that also gives us a lot of control over it. I have limited time to spend on this, so uploading the patch here for reference. Perhaps there is something there that could inspire you at least. I may get more time to revisit this next week or so. http://cr.openjdk.java.net/~erikj/windows-wsl/webrev.01/index.html /Erik On 2018-12-13 03:12, Magnus Ihse Bursie wrote: On 2018-12-13 08:48, Andrew Luo wrote: Hi, I attached the latest patch, which now can use Windows paths. Great! I looked at the code, and it looks really good. Just one request. I understand why you want to unify the similar code between msys and wsl, but unless you have actually verified on an msys system that this does not break anything -- please do not do it. This entire system of getting Windows environments to behave is very brittle, and even things that looks like they "should" work, often turn out not to do so in practice. So even though code duplication is a bad thing in general, in this case I'd rather see that you just added the support for WSL, instead of changing the old code. Unless, of course, you have verified that it does not break msys. I can also add that the code here is really horrendous. I'm sure there's a more efficient way of achieving what we want -- sane, space-free, universally usable paths that can be easily turned into windows paths by fixpath, but there's been too many corner-cases, too much of "oh no, now it breaks on cygwin if the co
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Oh, also, using WSLPATH to set PATH/l causes a LOT of extra output, namely, every time a Win32 executable is run this gets printed: <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/games <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/games <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /snap/bin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/sbin <3>init: (21845) ERROR: UtilTranslatePathList:2021: Failed to translate /usr/local/sbin Don't know if there's any way to tell WSL to suppress it. Thanks, -Andrew -Original Message- From: Andrew Luo Sent: Thursday, December 13, 2018 11:43 AM To: Erik Joelsson ; Magnus Ihse Bursie ; build-dev@openjdk.java.net Subject: RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hi Erik, Thanks for helping out on this. I made some changes to your patch and can get it working on my system. It's still building but it seems to be working. Will update this thread once it's finished building... Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Thursday, December 13, 2018 10:36 AM To: Magnus Ihse Bursie ; Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello, It's nice to see this progressing! I just wanted to let you know I took your patch from yesterday and started experimenting too. I managed to get configure to automatically find the Visual Studio installation as Magnus described, when running in a pure WSL shell without VS env. I still have some issues with the build though so the patch is not fully working yet. One thing I'm still experimenting with is the method of extraction of the VS variables. I think this can be improved more. In the old implementation we are relying on automatic conversion and sharing of some variables between Windows and the unix emulation (which Cygwin does for PATH and msys magically tries to do for all sorts of stuff). These need explicit declaration using WSLENV in WSL, but that also gives us a lot of control over it. I have limited time to spend on this, so uploading the patch here for reference. Perhaps there is something there that could inspire you at least. I may get more time to revisit this next week or so. http://cr.openjdk.java.net/~erikj/windows-wsl/webrev.01/index.html /Erik On 2018-12-13 03:12, Magnus Ihse Bursie wrote: > On 2018-12-13 08:48, Andrew Luo wrote: >> Hi, >> >> I attached the latest patch, which now can use Windows paths. > Great! > > I looked at the code, and it looks really good. Just one request. I > understand why you want to unify the similar code between msys and > wsl, but unless you have actually verified on an msys system that this > does not break anything -- please do not do it. This entire system of > getting Windows environments to behave is very brittle, and even > things that looks like they "should" work, often turn out not to do so > in practice. So even though code duplication is a bad thing in > general, in this case I'd rather see that you just added the support > for WSL, instead of changing the old code. Unless, of course, you have > verified that it does not break msys. > > I can also add that the code here is really horrendous. I'm sure > there's a more efficient way of achieving what we want -- sane, > space-free, universally usable paths that can be easily turned into > windows paths by fixpath, but there's been too many corner-cases, too > much of "oh no, now it breaks on cygwin if the code is in the users > home dir!" nasty surprises. Solving this properly would probably > involve creating some automated test that can be run on all two (soon > three) Windows unix environment. And I've never felt it worth it. So > it's been a lot of "well, just add this rewrite here too, just in > case, see, now it works!". I'm not proud of it, but it does it's thing. > > (I also note that you didn't care about tr:ing the 8.3 path to lower > case. It's of course a matter of taste, but since the goal is to > transform the path to a unix-style path, I tend to think that a path > like /mnt/c/p
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
or certain arguments, this might work anyway, out of pure chance. But it's not recommended. So if it ends up that you really need to point to the visual studio installation, the example in the build confiuration should be something like "--with-tools-dir="/mnt/c/Program Files (x86)/Microsoft Visual Studio/2017/Enterprise/VC/Auxiliary". The issues regarding the console input redirection I'm still investigating, but I doubt it's a bug in the build scripts, meaning it is likely a bug in the (boot) JDK or WSL. Even if we fix the JDK, we probably still have to support older boot JDKs (even if the patch is backported), and waiting for Microsoft to fix a bug in WSL could take a while (and might only be fixed in a later version of Windows). Thus, I think what we have is a good start, unless you think it's necessary to root cause the redirection issue first. That said, I will keep this thread updated with my progress once I figure out the root cause of the issue. No, it's not necessary to find out the root cause. It would be nice to know, but if the issue is only involving these two tools, and it happens deterministically if it happens, I'm fine. Especially since the workaround was actually an improvement. :-) /Magnus Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Wednesday, December 12, 2018 10:54 AM To: Erik Joelsson ; Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-12 18:30, Erik Joelsson wrote: Hello, I had the same trouble you describe trying to call cmd to create a short path from WSL with an inline script. I managed to it working by creating a script file like this: shortName.cmd: --- @ECHO OFF if '%1' NEQ '' echo %~s1 --- $ /mnt/c/Windows/System32/cmd.exe /c shortName.cmd "C:\Program Files" C:\PROGRA~1 We could put such a script in make/scripts. That's a clever workaround. Andrew, can you see if you can use that technique to get the proper space safe path resolution to work? I think you can copy the msys code straight as it is, and just replace the call to cmd echo %~sA with cmd /c $TOPDIR/make/script/shortName.cmd, or whatever you end up calling it. /Erik On 2018-12-11 22:44, Andrew Luo wrote: For the stdin/stdout redirection: basically, the issue was only occurring with those two executables. Oddly enough, it would occur every time I tried (for SPP the output would be cutoff, presumably because the input was cutoff, and for the other executable, available0 would throw an exception consistently for System.in). I have a hunch this is due to using WINAPI console functions for reading from a WSL shell, but I'm not 100% certain. I plan to try to build a smaller sample that can reproduce the issue, and if it seems to be a Windows bug I will file a bug with Microsoft. So what you are saying is that the issue was not intermittent, but always happened for those tools? If so, I can reluctantly agree to this fix. But I'd appreciate if you could drill down a bit and see what the problem really is. /Magnus As for the short paths, calling cmd.exe from WSL bash seems to be a bit buggy. cmd.exe, when called from a standard Windows environment, works properly and prints the path, however, when called from WSL, I get: "(C:\Program Files (x86))" was unexpected at this time. I verified that the correct path was being passed to cmd.exe (not a bash escaping issue) by creating my own test executable (C++) that just prints argv[0] ... argv[argc - 1] and when running my text executable from both Windows and WSL I get the same result, however when running cmd.exe with the exact same arguments, it works in Windows but not WSL. I will file a bug with Microsoft for this issue. Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Tuesday, December 11, 2018 6:18 AM To: Andrew Luo ; Erik Joelsson ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-11 14:37, Magnus Ihse Bursie wrote: On 2018-12-11 06:25, Andrew Luo wrote: Hi, Yes, I've signed an OCA (I've also contributed changes to other groups before, but not build). Okay, I have fixed the autconf-config* files. Unfortunately, as Erik mentioned, there is no (supported/reliable) way to access the WSL root / from /cygdrive/c, or even from Windows (there is a way in reality, however, it's not documented/supported by Microsoft and the location changes depending on the distribution/store app id/etc. so best to avoid using it.) I can see if we can print information about versions however. Right, WSL requires the .exe extension when accessing an executable, as this is Linux behavior (Linux doesn't have extensions for executables generally, but that's besides the point)... I fixed BASIC_REMOVE_SYMBOLIC_LINKS - a leftover from another approach I tried. For the
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
pens deterministically if it happens, I'm fine. Especially since the workaround was actually an improvement. :-) /Magnus Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Wednesday, December 12, 2018 10:54 AM To: Erik Joelsson ; Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-12 18:30, Erik Joelsson wrote: Hello, I had the same trouble you describe trying to call cmd to create a short path from WSL with an inline script. I managed to it working by creating a script file like this: shortName.cmd: --- @ECHO OFF if '%1' NEQ '' echo %~s1 --- $ /mnt/c/Windows/System32/cmd.exe /c shortName.cmd "C:\Program Files" C:\PROGRA~1 We could put such a script in make/scripts. That's a clever workaround. Andrew, can you see if you can use that technique to get the proper space safe path resolution to work? I think you can copy the msys code straight as it is, and just replace the call to cmd echo %~sA with cmd /c $TOPDIR/make/script/shortName.cmd, or whatever you end up calling it. /Erik On 2018-12-11 22:44, Andrew Luo wrote: For the stdin/stdout redirection: basically, the issue was only occurring with those two executables. Oddly enough, it would occur every time I tried (for SPP the output would be cutoff, presumably because the input was cutoff, and for the other executable, available0 would throw an exception consistently for System.in). I have a hunch this is due to using WINAPI console functions for reading from a WSL shell, but I'm not 100% certain. I plan to try to build a smaller sample that can reproduce the issue, and if it seems to be a Windows bug I will file a bug with Microsoft. So what you are saying is that the issue was not intermittent, but always happened for those tools? If so, I can reluctantly agree to this fix. But I'd appreciate if you could drill down a bit and see what the problem really is. /Magnus As for the short paths, calling cmd.exe from WSL bash seems to be a bit buggy. cmd.exe, when called from a standard Windows environment, works properly and prints the path, however, when called from WSL, I get: "(C:\Program Files (x86))" was unexpected at this time. I verified that the correct path was being passed to cmd.exe (not a bash escaping issue) by creating my own test executable (C++) that just prints argv[0] ... argv[argc - 1] and when running my text executable from both Windows and WSL I get the same result, however when running cmd.exe with the exact same arguments, it works in Windows but not WSL. I will file a bug with Microsoft for this issue. Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Tuesday, December 11, 2018 6:18 AM To: Andrew Luo ; Erik Joelsson ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-11 14:37, Magnus Ihse Bursie wrote: On 2018-12-11 06:25, Andrew Luo wrote: Hi, Yes, I've signed an OCA (I've also contributed changes to other groups before, but not build). Okay, I have fixed the autconf-config* files. Unfortunately, as Erik mentioned, there is no (supported/reliable) way to access the WSL root / from /cygdrive/c, or even from Windows (there is a way in reality, however, it's not documented/supported by Microsoft and the location changes depending on the distribution/store app id/etc. so best to avoid using it.) I can see if we can print information about versions however. Right, WSL requires the .exe extension when accessing an executable, as this is Linux behavior (Linux doesn't have extensions for executables generally, but that's besides the point)... I fixed BASIC_REMOVE_SYMBOLIC_LINKS - a leftover from another approach I tried. For the redirect, redirect doesn't seem to be working when you have a bash shell input piped into a Win32 executable reading from stdin using WINAPI. I'm not sure this is supported by the OpenJDK, more likely it might be a Microsoft issue. For some reason, the stdin would be cut off (or I would see an exception thrown from available0 in FileInputStream). I personally didn't see any harm in changing piping into input/output files (since all the inputs/outputs are files anyways!). Ok, let me be sure I get this right. It is only the redirect of *input* that fails? (But you fixed both because of consistency). I agree that the change itself is fine, even better than it is right now -- I was mostly worried about the consequences of redirects is not working; there might be other places that fail. But if redirecting output works, I think we're mostly fine. That's something we do all the time, for each executed command, so if that did not work reliably it would be really bad. But still... I tried greping for "<" and there's a lot of places, 20+, that redirects input. Or did this problem only happen when running *java* as the recipient of the red
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hi, I attached the latest patch, which now can use Windows paths. The new instructions to build are (assuming 8.3 paths are enabled on your system...): 1. wsl must be started from a Windows Developer command prompt. To ensure the correct environment variables are propagated from Windows to WSL, you can run the following commands: set WSLENV=INCLUDE/l:LIBPATH/l 2. Start wsl (bash): wsl 3. Run configure: ./configure --with-boot-jdk=/mnt/c/Users/Andrew/Downloads/openjdk-11.0.1_windows-x64_bin/jdk-11.0.1 --with-tools-dir="C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary" --with-ucrt-dll-dir="C:\Program Files (x86)\Windows Kits\10\Redist\ucrt\DLLs\x64" 4. Run make I’ve tested make with the default target as well as “make images” The issues regarding the console input redirection I'm still investigating, but I doubt it's a bug in the build scripts, meaning it is likely a bug in the (boot) JDK or WSL. Even if we fix the JDK, we probably still have to support older boot JDKs (even if the patch is backported), and waiting for Microsoft to fix a bug in WSL could take a while (and might only be fixed in a later version of Windows). Thus, I think what we have is a good start, unless you think it's necessary to root cause the redirection issue first. That said, I will keep this thread updated with my progress once I figure out the root cause of the issue. Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Wednesday, December 12, 2018 10:54 AM To: Erik Joelsson ; Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-12 18:30, Erik Joelsson wrote: > Hello, > > I had the same trouble you describe trying to call cmd to create a > short path from WSL with an inline script. I managed to it working by > creating a script file like this: > > shortName.cmd: > > --- > @ECHO OFF > if '%1' NEQ '' echo %~s1 > --- > > $ /mnt/c/Windows/System32/cmd.exe /c shortName.cmd "C:\Program Files" > C:\PROGRA~1 > > We could put such a script in make/scripts. That's a clever workaround. Andrew, can you see if you can use that technique to get the proper space safe path resolution to work? I think you can copy the msys code straight as it is, and just replace the call to cmd echo %~sA with cmd /c $TOPDIR/make/script/shortName.cmd, or whatever you end up calling it. > > /Erik > > On 2018-12-11 22:44, Andrew Luo wrote: >> For the stdin/stdout redirection: basically, the issue was only >> occurring with those two executables. Oddly enough, it would occur >> every time I tried (for SPP the output would be cutoff, presumably >> because the input was cutoff, and for the other executable, >> available0 would throw an exception consistently for System.in). I >> have a hunch this is due to using WINAPI console functions for >> reading from a WSL shell, but I'm not 100% certain. I plan to try to >> build a smaller sample that can reproduce the issue, and if it seems >> to be a Windows bug I will file a bug with Microsoft. So what you are saying is that the issue was not intermittent, but always happened for those tools? If so, I can reluctantly agree to this fix. But I'd appreciate if you could drill down a bit and see what the problem really is. /Magnus >> >> As for the short paths, calling cmd.exe from WSL bash seems to be a >> bit buggy. cmd.exe, when called from a standard Windows environment, >> works properly and prints the path, however, when called from WSL, I >> get: >> >> "(C:\Program Files (x86))" was unexpected at this time. >> >> I verified that the correct path was being passed to cmd.exe (not a >> bash escaping issue) by creating my own test executable (C++) that >> just prints argv[0] ... argv[argc - 1] and when running my text >> executable from both Windows and WSL I get the same result, however >> when running cmd.exe with the exact same arguments, it works in >> Windows but not WSL. I will file a bug with Microsoft for this issue. >> >> Thanks, >> >> -Andrew >> >> >> -Original Message- >> From: Magnus Ihse Bursie >> Sent: Tuesday, December 11, 2018 6:18 AM >> To: Andrew Luo ; Erik Joelsson >> ; build-dev@openjdk.java.net >> Subject: Re: [PATCH] Support for building using WSL (Windows >> Subsystem for Linux) on Windows >> >> >> >> On 2018-12-11 14:37, Magnus Ihse Bursie wrote: >>> On 2018-12-11 06:25, Andrew Luo wrote: >>>> Hi, >>>> >>>> Yes, I've signed an OCA (I've also contributed changes to other >>>> groups before, b
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
On 2018-12-12 18:30, Erik Joelsson wrote: Hello, I had the same trouble you describe trying to call cmd to create a short path from WSL with an inline script. I managed to it working by creating a script file like this: shortName.cmd: --- @ECHO OFF if '%1' NEQ '' echo %~s1 --- $ /mnt/c/Windows/System32/cmd.exe /c shortName.cmd "C:\Program Files" C:\PROGRA~1 We could put such a script in make/scripts. That's a clever workaround. Andrew, can you see if you can use that technique to get the proper space safe path resolution to work? I think you can copy the msys code straight as it is, and just replace the call to cmd echo %~sA with cmd /c $TOPDIR/make/script/shortName.cmd, or whatever you end up calling it. /Erik On 2018-12-11 22:44, Andrew Luo wrote: For the stdin/stdout redirection: basically, the issue was only occurring with those two executables. Oddly enough, it would occur every time I tried (for SPP the output would be cutoff, presumably because the input was cutoff, and for the other executable, available0 would throw an exception consistently for System.in). I have a hunch this is due to using WINAPI console functions for reading from a WSL shell, but I'm not 100% certain. I plan to try to build a smaller sample that can reproduce the issue, and if it seems to be a Windows bug I will file a bug with Microsoft. So what you are saying is that the issue was not intermittent, but always happened for those tools? If so, I can reluctantly agree to this fix. But I'd appreciate if you could drill down a bit and see what the problem really is. /Magnus As for the short paths, calling cmd.exe from WSL bash seems to be a bit buggy. cmd.exe, when called from a standard Windows environment, works properly and prints the path, however, when called from WSL, I get: "(C:\Program Files (x86))" was unexpected at this time. I verified that the correct path was being passed to cmd.exe (not a bash escaping issue) by creating my own test executable (C++) that just prints argv[0] ... argv[argc - 1] and when running my text executable from both Windows and WSL I get the same result, however when running cmd.exe with the exact same arguments, it works in Windows but not WSL. I will file a bug with Microsoft for this issue. Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Tuesday, December 11, 2018 6:18 AM To: Andrew Luo ; Erik Joelsson ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-11 14:37, Magnus Ihse Bursie wrote: On 2018-12-11 06:25, Andrew Luo wrote: Hi, Yes, I've signed an OCA (I've also contributed changes to other groups before, but not build). Okay, I have fixed the autconf-config* files. Unfortunately, as Erik mentioned, there is no (supported/reliable) way to access the WSL root / from /cygdrive/c, or even from Windows (there is a way in reality, however, it's not documented/supported by Microsoft and the location changes depending on the distribution/store app id/etc. so best to avoid using it.) I can see if we can print information about versions however. Right, WSL requires the .exe extension when accessing an executable, as this is Linux behavior (Linux doesn't have extensions for executables generally, but that's besides the point)... I fixed BASIC_REMOVE_SYMBOLIC_LINKS - a leftover from another approach I tried. For the redirect, redirect doesn't seem to be working when you have a bash shell input piped into a Win32 executable reading from stdin using WINAPI. I'm not sure this is supported by the OpenJDK, more likely it might be a Microsoft issue. For some reason, the stdin would be cut off (or I would see an exception thrown from available0 in FileInputStream). I personally didn't see any harm in changing piping into input/output files (since all the inputs/outputs are files anyways!). Ok, let me be sure I get this right. It is only the redirect of *input* that fails? (But you fixed both because of consistency). I agree that the change itself is fine, even better than it is right now -- I was mostly worried about the consequences of redirects is not working; there might be other places that fail. But if redirecting output works, I think we're mostly fine. That's something we do all the time, for each executed command, so if that did not work reliably it would be really bad. But still... I tried greping for "<" and there's a lot of places, 20+, that redirects input. Or did this problem only happen when running *java* as the recipient of the redirected input? This worries me, and while I do think your change makes the tools have a better UI, I don't like this as a workaround that will not solve all potential problems. :( I fixed TOOLCHAIN_FIND_VISUAL_STUDIO_BAT_FILE - this was also from a few things I had tried earlier. I disabled the $BASH code because to call bash from Win32 the cor
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hello, I had the same trouble you describe trying to call cmd to create a short path from WSL with an inline script. I managed to it working by creating a script file like this: shortName.cmd: --- @ECHO OFF if '%1' NEQ '' echo %~s1 --- $ /mnt/c/Windows/System32/cmd.exe /c shortName.cmd "C:\Program Files" C:\PROGRA~1 We could put such a script in make/scripts. /Erik On 2018-12-11 22:44, Andrew Luo wrote: For the stdin/stdout redirection: basically, the issue was only occurring with those two executables. Oddly enough, it would occur every time I tried (for SPP the output would be cutoff, presumably because the input was cutoff, and for the other executable, available0 would throw an exception consistently for System.in). I have a hunch this is due to using WINAPI console functions for reading from a WSL shell, but I'm not 100% certain. I plan to try to build a smaller sample that can reproduce the issue, and if it seems to be a Windows bug I will file a bug with Microsoft. As for the short paths, calling cmd.exe from WSL bash seems to be a bit buggy. cmd.exe, when called from a standard Windows environment, works properly and prints the path, however, when called from WSL, I get: "(C:\Program Files (x86))" was unexpected at this time. I verified that the correct path was being passed to cmd.exe (not a bash escaping issue) by creating my own test executable (C++) that just prints argv[0] ... argv[argc - 1] and when running my text executable from both Windows and WSL I get the same result, however when running cmd.exe with the exact same arguments, it works in Windows but not WSL. I will file a bug with Microsoft for this issue. Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Tuesday, December 11, 2018 6:18 AM To: Andrew Luo ; Erik Joelsson ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-11 14:37, Magnus Ihse Bursie wrote: On 2018-12-11 06:25, Andrew Luo wrote: Hi, Yes, I've signed an OCA (I've also contributed changes to other groups before, but not build). Okay, I have fixed the autconf-config* files. Unfortunately, as Erik mentioned, there is no (supported/reliable) way to access the WSL root / from /cygdrive/c, or even from Windows (there is a way in reality, however, it's not documented/supported by Microsoft and the location changes depending on the distribution/store app id/etc. so best to avoid using it.) I can see if we can print information about versions however. Right, WSL requires the .exe extension when accessing an executable, as this is Linux behavior (Linux doesn't have extensions for executables generally, but that's besides the point)... I fixed BASIC_REMOVE_SYMBOLIC_LINKS - a leftover from another approach I tried. For the redirect, redirect doesn't seem to be working when you have a bash shell input piped into a Win32 executable reading from stdin using WINAPI. I'm not sure this is supported by the OpenJDK, more likely it might be a Microsoft issue. For some reason, the stdin would be cut off (or I would see an exception thrown from available0 in FileInputStream). I personally didn't see any harm in changing piping into input/output files (since all the inputs/outputs are files anyways!). Ok, let me be sure I get this right. It is only the redirect of *input* that fails? (But you fixed both because of consistency). I agree that the change itself is fine, even better than it is right now -- I was mostly worried about the consequences of redirects is not working; there might be other places that fail. But if redirecting output works, I think we're mostly fine. That's something we do all the time, for each executed command, so if that did not work reliably it would be really bad. But still... I tried greping for "<" and there's a lot of places, 20+, that redirects input. Or did this problem only happen when running *java* as the recipient of the redirected input? This worries me, and while I do think your change makes the tools have a better UI, I don't like this as a workaround that will not solve all potential problems. :( I fixed TOOLCHAIN_FIND_VISUAL_STUDIO_BAT_FILE - this was also from a few things I had tried earlier. I disabled the $BASH code because to call bash from Win32 the correct way is either "wsl /bin/bash" or just "bash". $BASH correctly evaluates to /bin/bash, however BASIC_WINDOWS_REWRITE_AS_WINDOWS_MIXED_PATH is implemented in terms of wslpath, which can only convert a path under /mnt/c back to a Windows path. Other files under /, for example /bin and /bin/bash, cannot be converted to a Windwos path. The escaping changes I made because it wasn't working. This does work with spaces in the path on WSL. I don't have a Cygwin environment to check, perhaps someone else here could help out? Otherwise I can refactor that code to use that echo statem
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
For the stdin/stdout redirection: basically, the issue was only occurring with those two executables. Oddly enough, it would occur every time I tried (for SPP the output would be cutoff, presumably because the input was cutoff, and for the other executable, available0 would throw an exception consistently for System.in). I have a hunch this is due to using WINAPI console functions for reading from a WSL shell, but I'm not 100% certain. I plan to try to build a smaller sample that can reproduce the issue, and if it seems to be a Windows bug I will file a bug with Microsoft. As for the short paths, calling cmd.exe from WSL bash seems to be a bit buggy. cmd.exe, when called from a standard Windows environment, works properly and prints the path, however, when called from WSL, I get: "(C:\Program Files (x86))" was unexpected at this time. I verified that the correct path was being passed to cmd.exe (not a bash escaping issue) by creating my own test executable (C++) that just prints argv[0] ... argv[argc - 1] and when running my text executable from both Windows and WSL I get the same result, however when running cmd.exe with the exact same arguments, it works in Windows but not WSL. I will file a bug with Microsoft for this issue. Thanks, -Andrew -Original Message- From: Magnus Ihse Bursie Sent: Tuesday, December 11, 2018 6:18 AM To: Andrew Luo ; Erik Joelsson ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows On 2018-12-11 14:37, Magnus Ihse Bursie wrote: > On 2018-12-11 06:25, Andrew Luo wrote: >> Hi, >> >> Yes, I've signed an OCA (I've also contributed changes to other >> groups before, but not build). >> >> Okay, I have fixed the autconf-config* files. >> >> Unfortunately, as Erik mentioned, there is no (supported/reliable) >> way to access the WSL root / from /cygdrive/c, or even from Windows >> (there is a way in reality, however, it's not documented/supported by >> Microsoft and the location changes depending on the >> distribution/store app id/etc. so best to avoid using it.) I can see >> if we can print information about versions however. >> >> Right, WSL requires the .exe extension when accessing an executable, >> as this is Linux behavior (Linux doesn't have extensions for >> executables generally, but that's besides the point)... >> >> I fixed BASIC_REMOVE_SYMBOLIC_LINKS - a leftover from another >> approach I tried. >> >> For the redirect, redirect doesn't seem to be working when you have a >> bash shell input piped into a Win32 executable reading from stdin >> using WINAPI. I'm not sure this is supported by the OpenJDK, more >> likely it might be a Microsoft issue. For some reason, the stdin >> would be cut off (or I would see an exception thrown from available0 >> in FileInputStream). I personally didn't see any harm in changing >> piping into input/output files (since all the inputs/outputs are >> files anyways!). > Ok, let me be sure I get this right. It is only the redirect of > *input* that fails? (But you fixed both because of consistency). I > agree that the change itself is fine, even better than it is right now > -- I was mostly worried about the consequences of redirects is not > working; there might be other places that fail. But if redirecting > output works, I think we're mostly fine. That's something we do all > the time, for each executed command, so if that did not work reliably > it would be really bad. > > But still... I tried greping for "<" and there's a lot of places, 20+, > that redirects input. > > Or did this problem only happen when running *java* as the recipient > of the redirected input? > > This worries me, and while I do think your change makes the tools have > a better UI, I don't like this as a workaround that will not solve all > potential problems. :( > >> >> I fixed TOOLCHAIN_FIND_VISUAL_STUDIO_BAT_FILE - this was also from a >> few things I had tried earlier. >> >> I disabled the $BASH code because to call bash from Win32 the correct >> way is either "wsl /bin/bash" or just "bash". $BASH correctly >> evaluates to /bin/bash, however >> BASIC_WINDOWS_REWRITE_AS_WINDOWS_MIXED_PATH is implemented in terms >> of wslpath, which can only convert a path under /mnt/c back to a >> Windows path. Other files under /, for example /bin and /bin/bash, >> cannot be converted to a Windwos path. >> >> The escaping changes I made because it wasn't working. This does >> work with spaces in the path on WSL. I don't have a Cygwin >> environment to check, perhaps som
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
the "%~sA" variable syntax to print the 8.3 version of the path (input_path is a "normal" Windows path). Is there any way it's possible to do this on WSL? It seems reasonable that you should be able to call cmd.exe and redirect the output. I think it will be worth trying to jump through some loops or doing some dirty tricks to get this to work, because everything will be *s* much simpler if you can reliably turn paths into space-safe paths; our normal Windows build depends on it. I also realized that if you make a WSL version of BASIC_FIXUP_EXECUTABLE, then you might be able to add .exe in it. You will still need the EXE_SUFFIX when e.g. looking for the java.exe binary in locating the Boot JDK, but perhaps it will simplify some of the places. I see now that the call to java in Images.gmk really should have been prefixed with $(FIXPATH) instead. Then you would not have needed the EXE_SUFFIX fix there. Also, I suggest you look more closely on how we do things on msys than on cygwin; I have the feeling wsl is closer to msys (in terms of functionality from our perspective) than cygwin. /Magnus Attaching my latest patch (generated using powershell, so to properly import you may need to convert UTF16 -> UTF8 and change CRLF to just LF, hg tends to be picky)... Looks much better now! Before accepting it, I'd like to understand more about the redirect issue, and see if there really is no way to rewrite the paths in a space-safe manner. Then I think this is good to go. /Magnus Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 10, 2018 9:19 AM To: Magnus Ihse Bursie ; Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello, On 2018-12-10 02:06, Magnus Ihse Bursie wrote: On 2018-12-09 20:11, Andrew Luo wrote: One important thing to note is that the WSL build targets Windows. It is also possible to use WSL to target itself (a WSL Linux binary) or even other distributions of Linux. I have not implemented that yet, but I think I could do that as a next step if you guys think it would be useful (at least I think it would be useful, then you can test your changes for both Windows and Linux on one system...). I think if you just run configure ordinarily, it will behave like a Linux system and build the Linux image right out-of-the-box..? But then again, maybe that behavior is negated by your changes to config.guess and platform.m4. So maybe we need a flag to configure to control this... It is indeed possible to build a pure Linux binary in WSL today so I think it would be bad to lose that functionality. We certainly need a configure flag to control if a Windows or Linux build should be produced in this case. This is something I have been thinking about when I started tackling WSL builds some time ago but didn't really come up with a good solution. I didn't have the time to spend to really see it through though, so it's nice to see that someone else is trying. We could simply use the --with-openjdk-target, that would perhaps be the cleanest, but it's also a bit cumbersome. We may need some simplification similar to how we have --with-target-bits=32/64 as a simple switch (e.g. --with-wsl-target=linux/windows?). Steps in case you want to try this out: 1. Due to autotools not handling spaces well, you have to create symlinks in Windows that will allow you to access Windows Kits and the VC++ compiler without spaces in the path: mklink /D C:\VS "C:\Program Files (x86)\Microsoft Visual Studio" mklink /D C:\WindowsKits "C:\Program Files (x86)\Windows Kits" That's a bit odd. We encounter spaces in paths on Windows normally on cygwin and msys, and that works fine. I suspect there is something missing with the rewriting functions. What we do, is that we rewrite paths with spaces to paths without spaces, by using the old 8+3 compatibility names, so we get something like "/cygdrive/c/progra~1/microso~2" from "C:\Program Files (x86)\Microsoft Visual Studio". Have a look at BASIC_MAKE_WINDOWS_SPACE_SAFE_CYGWIN. I think you need a WSL version of that, as well as of BASIC_FIXUP_PATH_CYGWIN. (And you need to call the BASIC_FIXUP_PATH_WSL from BASIC_FIXUP_PATH.) If you get these parts right, I don't think you will need any of the special instructions below to build. In fact, as long as C:\... is properly remapped, the normal VS autodetect code should work just fine. And perhaps you can even revert some of the scarier changes in toolchain_windows.m4. I definitely agree with Magnus that to make WSL truly supported, the path handling macros need to be replicated. I'm not sure how to solve it properly. The root path Magnus is asking for is not defined in WSL. In fact, from windows you cannot reach any path in the WSL filesystem. Only Windows drives are mounted in WSL, not the other way
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
(input_path is a "normal" Windows path). Is there any way it's possible to do this on WSL? It seems reasonable that you should be able to call cmd.exe and redirect the output. I think it will be worth trying to jump through some loops or doing some dirty tricks to get this to work, because everything will be *s* much simpler if you can reliably turn paths into space-safe paths; our normal Windows build depends on it. /Magnus Attaching my latest patch (generated using powershell, so to properly import you may need to convert UTF16 -> UTF8 and change CRLF to just LF, hg tends to be picky)... Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 10, 2018 9:19 AM To: Magnus Ihse Bursie ; Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello, On 2018-12-10 02:06, Magnus Ihse Bursie wrote: On 2018-12-09 20:11, Andrew Luo wrote: One important thing to note is that the WSL build targets Windows. It is also possible to use WSL to target itself (a WSL Linux binary) or even other distributions of Linux. I have not implemented that yet, but I think I could do that as a next step if you guys think it would be useful (at least I think it would be useful, then you can test your changes for both Windows and Linux on one system...). I think if you just run configure ordinarily, it will behave like a Linux system and build the Linux image right out-of-the-box..? But then again, maybe that behavior is negated by your changes to config.guess and platform.m4. So maybe we need a flag to configure to control this... It is indeed possible to build a pure Linux binary in WSL today so I think it would be bad to lose that functionality. We certainly need a configure flag to control if a Windows or Linux build should be produced in this case. This is something I have been thinking about when I started tackling WSL builds some time ago but didn't really come up with a good solution. I didn't have the time to spend to really see it through though, so it's nice to see that someone else is trying. We could simply use the --with-openjdk-target, that would perhaps be the cleanest, but it's also a bit cumbersome. We may need some simplification similar to how we have --with-target-bits=32/64 as a simple switch (e.g. --with-wsl-target=linux/windows?). Steps in case you want to try this out: 1. Due to autotools not handling spaces well, you have to create symlinks in Windows that will allow you to access Windows Kits and the VC++ compiler without spaces in the path: mklink /D C:\VS "C:\Program Files (x86)\Microsoft Visual Studio" mklink /D C:\WindowsKits "C:\Program Files (x86)\Windows Kits" That's a bit odd. We encounter spaces in paths on Windows normally on cygwin and msys, and that works fine. I suspect there is something missing with the rewriting functions. What we do, is that we rewrite paths with spaces to paths without spaces, by using the old 8+3 compatibility names, so we get something like "/cygdrive/c/progra~1/microso~2" from "C:\Program Files (x86)\Microsoft Visual Studio". Have a look at BASIC_MAKE_WINDOWS_SPACE_SAFE_CYGWIN. I think you need a WSL version of that, as well as of BASIC_FIXUP_PATH_CYGWIN. (And you need to call the BASIC_FIXUP_PATH_WSL from BASIC_FIXUP_PATH.) If you get these parts right, I don't think you will need any of the special instructions below to build. In fact, as long as C:\... is properly remapped, the normal VS autodetect code should work just fine. And perhaps you can even revert some of the scarier changes in toolchain_windows.m4. I definitely agree with Magnus that to make WSL truly supported, the path handling macros need to be replicated. I'm not sure how to solve it properly. The root path Magnus is asking for is not defined in WSL. In fact, from windows you cannot reach any path in the WSL filesystem. Only Windows drives are mounted in WSL, not the other way around. To convert to old style paths in Cygwin we rely on the cygpath utility. There is a wslpath utility but does it support old style path conversions? If not, maybe it's possible to write such a tool in CMD/PowerShell? /Erik 2. wsl must be started from a Windows Developer command prompt. To ensure the correct environment variables are propagated from Windows to WSL, you can run the following commands: set WSLENV=INCLUDE/l:LIBPATH/l 3. Start wsl (bash): wsl 4. After starting bash you must set your compiler variables to explicitly point to the correct tools: export AR=/mnt/c/VS/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/lib.exe export CC=/mnt/c/VS/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe export CXX=/mnt/c/VS/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe export LD=/mnt/c/VS/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Ho
RE: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hi, Yes, I've signed an OCA (I've also contributed changes to other groups before, but not build). Okay, I have fixed the autconf-config* files. Unfortunately, as Erik mentioned, there is no (supported/reliable) way to access the WSL root / from /cygdrive/c, or even from Windows (there is a way in reality, however, it's not documented/supported by Microsoft and the location changes depending on the distribution/store app id/etc. so best to avoid using it.) I can see if we can print information about versions however. Right, WSL requires the .exe extension when accessing an executable, as this is Linux behavior (Linux doesn't have extensions for executables generally, but that's besides the point)... I fixed BASIC_REMOVE_SYMBOLIC_LINKS - a leftover from another approach I tried. For the redirect, redirect doesn't seem to be working when you have a bash shell input piped into a Win32 executable reading from stdin using WINAPI. I'm not sure this is supported by the OpenJDK, more likely it might be a Microsoft issue. For some reason, the stdin would be cut off (or I would see an exception thrown from available0 in FileInputStream). I personally didn't see any harm in changing piping into input/output files (since all the inputs/outputs are files anyways!). I fixed TOOLCHAIN_FIND_VISUAL_STUDIO_BAT_FILE - this was also from a few things I had tried earlier. I disabled the $BASH code because to call bash from Win32 the correct way is either "wsl /bin/bash" or just "bash". $BASH correctly evaluates to /bin/bash, however BASIC_WINDOWS_REWRITE_AS_WINDOWS_MIXED_PATH is implemented in terms of wslpath, which can only convert a path under /mnt/c back to a Windows path. Other files under /, for example /bin and /bin/bash, cannot be converted to a Windwos path. The escaping changes I made because it wasn't working. This does work with spaces in the path on WSL. I don't have a Cygwin environment to check, perhaps someone else here could help out? Otherwise I can refactor that code to use that echo statement for WSL and use the old echo statement for Cygwin. I have fixed the extraneous debug print statement. As for Windows vs Linux output - you can still force it to build a Linux output binary. You just need to run configure as follows: ./configure --with-boot-jdk=/home/andrew/jdk-11.0.1 build=x86_64-unknown-linux-gnu --host=x86_64-unknown-linux-gnu However, there is a behavior change: now, on WSL, by default, Windows binaries are targeted. Previously, Linux binaries were the default target. (Also, you can run configure twice and two sets of configurations will be generated, you can actually build both images by setting CONF=linux-x86_64-server-release or CONF=windows-x86_64-server-release) As for BASIC_MAKE_WINDOWS_SPACE_SAFE_CYGWIN, wslpath does not support 8.3 names. But perhaps the symlink workaround is acceptable for now and we can handle the 8.3 naming on WSL in a separate change, what do you guys think - personally I think what we have (assuming Cygwin still works) is at least a MVP for WSL devs. Anyways, at least some people may have to use the symlink workaround if they've disabled 8.3 on NTFS. Attaching my latest patch (generated using powershell, so to properly import you may need to convert UTF16 -> UTF8 and change CRLF to just LF, hg tends to be picky)... Thanks, -Andrew -Original Message- From: Erik Joelsson Sent: Monday, December 10, 2018 9:19 AM To: Magnus Ihse Bursie ; Andrew Luo ; build-dev@openjdk.java.net Subject: Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows Hello, On 2018-12-10 02:06, Magnus Ihse Bursie wrote: > > On 2018-12-09 20:11, Andrew Luo wrote: >> >> One important thing to note is that the WSL build targets Windows. It >> is also possible to use WSL to target itself (a WSL Linux binary) or >> even other distributions of Linux. I have not implemented that yet, >> but I think I could do that as a next step if you guys think it would >> be useful (at least I think it would be useful, then you can test >> your changes for both Windows and Linux on one system...). > I think if you just run configure ordinarily, it will behave like a > Linux system and build the Linux image right out-of-the-box..? But > then again, maybe that behavior is negated by your changes to > config.guess and platform.m4. So maybe we need a flag to configure to > control this... It is indeed possible to build a pure Linux binary in WSL today so I think it would be bad to lose that functionality. We certainly need a configure flag to control if a Windows or Linux build should be produced in this case. This is something I have been thinking about when I started tackling WSL builds some time ago but didn't really come up with a good solution. I didn't have the time to spend to really see it
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hello, On 2018-12-10 02:06, Magnus Ihse Bursie wrote: On 2018-12-09 20:11, Andrew Luo wrote: One important thing to note is that the WSL build targets Windows. It is also possible to use WSL to target itself (a WSL Linux binary) or even other distributions of Linux. I have not implemented that yet, but I think I could do that as a next step if you guys think it would be useful (at least I think it would be useful, then you can test your changes for both Windows and Linux on one system...). I think if you just run configure ordinarily, it will behave like a Linux system and build the Linux image right out-of-the-box..? But then again, maybe that behavior is negated by your changes to config.guess and platform.m4. So maybe we need a flag to configure to control this... It is indeed possible to build a pure Linux binary in WSL today so I think it would be bad to lose that functionality. We certainly need a configure flag to control if a Windows or Linux build should be produced in this case. This is something I have been thinking about when I started tackling WSL builds some time ago but didn't really come up with a good solution. I didn't have the time to spend to really see it through though, so it's nice to see that someone else is trying. We could simply use the --with-openjdk-target, that would perhaps be the cleanest, but it's also a bit cumbersome. We may need some simplification similar to how we have --with-target-bits=32/64 as a simple switch (e.g. --with-wsl-target=linux/windows?). Steps in case you want to try this out: 1. Due to autotools not handling spaces well, you have to create symlinks in Windows that will allow you to access Windows Kits and the VC++ compiler without spaces in the path: mklink /D C:\VS "C:\Program Files (x86)\Microsoft Visual Studio" mklink /D C:\WindowsKits "C:\Program Files (x86)\Windows Kits" That's a bit odd. We encounter spaces in paths on Windows normally on cygwin and msys, and that works fine. I suspect there is something missing with the rewriting functions. What we do, is that we rewrite paths with spaces to paths without spaces, by using the old 8+3 compatibility names, so we get something like "/cygdrive/c/progra~1/microso~2" from "C:\Program Files (x86)\Microsoft Visual Studio". Have a look at BASIC_MAKE_WINDOWS_SPACE_SAFE_CYGWIN. I think you need a WSL version of that, as well as of BASIC_FIXUP_PATH_CYGWIN. (And you need to call the BASIC_FIXUP_PATH_WSL from BASIC_FIXUP_PATH.) If you get these parts right, I don't think you will need any of the special instructions below to build. In fact, as long as C:\... is properly remapped, the normal VS autodetect code should work just fine. And perhaps you can even revert some of the scarier changes in toolchain_windows.m4. I definitely agree with Magnus that to make WSL truly supported, the path handling macros need to be replicated. I'm not sure how to solve it properly. The root path Magnus is asking for is not defined in WSL. In fact, from windows you cannot reach any path in the WSL filesystem. Only Windows drives are mounted in WSL, not the other way around. To convert to old style paths in Cygwin we rely on the cygpath utility. There is a wslpath utility but does it support old style path conversions? If not, maybe it's possible to write such a tool in CMD/PowerShell? /Erik 2. wsl must be started from a Windows Developer command prompt. To ensure the correct environment variables are propagated from Windows to WSL, you can run the following commands: set WSLENV=INCLUDE/l:LIBPATH/l 3. Start wsl (bash): wsl 4. After starting bash you must set your compiler variables to explicitly point to the correct tools: export AR=/mnt/c/VS/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/lib.exe export CC=/mnt/c/VS/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe export CXX=/mnt/c/VS/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe export LD=/mnt/c/VS/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/link.exe export RC=/mnt/c/WindowsKits/10/bin/10.0.17763.0/x64/rc.exe export MT=/mnt/c/WindowsKits/10/bin/10.0.17763.0/x64/mt.exe export DUMPBIN=/mnt/c/VS/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/dumpbin.exe 5. Run configure: ./configure --with-boot-jdk=/mnt/c/Users/Andrew/Downloads/openjdk-11.0.1_windows-x64_bin/jdk-11.0.1 --with-tools-dir="C:\VS\2017\Enterprise\VC\Auxiliary" --with-ucrt-dll-dir="/mnt/c/WindowsKits/10/Redist/ucrt/DLLs/x64" 6. Run make I've tested make with the default target as well as "make images" Let me know if you have any feedback/comments. Thanks, -Andrew
Re: [PATCH] Support for building using WSL (Windows Subsystem for Linux) on Windows
Hi Andrew, Interesting! We've been thinking about adding support for WSL for some time now, but never gotten around to it. It's really appreciated to get your help here. Just a check, have you signed the OCA? Overall, I think your patch looks clean and well written! I still have some comments/questions, though: * Due to license issues, we cannot change the autoconf-config.* files. That's why we have the two wrapper files, that adjust the input/output to the corresponding original autoconf file. * For cygwin/msys we do this extra detection step in basics_windows.m4 where we try to locate and describe the "unix" root directory in terms of a fully qualified unix path, that is, something like /cygdrive/c/cygwin64. I'm not sure if it's actively used anymore, but it's printed in the configure log, and I've found it very helpful when analyzing problems given a log. I don't know the details of how WSL work, but if it's similar (e.g. the root of the linux file system can be expressed as a path using mounts), I'd appreciate if it could be printed, too. Also, the cygwin/msys version is printed; I'm not sure about the possibilities here, but perhaps the relevant information is the Windows version, lsb_base -a for the virtual linux, and possibly a WSL version number, if there is such a thing..? * It's a bit sad that you have to specify the .exe all over the place. :( But not much to do about it, I guess. I'm suspecting that there are still places you have missed to add the $EXECUTABLE_SUFFIX, but they will probably turn up quickly once this starts getting used for real. * Was BASIC_REMOVE_SYMBOLIC_LINKS not working properly? Maybe it's better to try and fix it, than work around it... * What's the deal with changing the redirect for TOOL_GENERATECURRENCYDATA and TOOL_SPP? Did redirects not work? That worries me a bit, since we have lots of other places with redirects. * I'm not sure why you put part of the code in TOOLCHAIN_FIND_VISUAL_STUDIO_BAT_FILE into an else clause. The idea is that TOOLCHAIN_CHECK_POSSIBLE_VISUAL_STUDIO_ROOT will do nothing if we've already located VS, so to avoid a long chain of if ... elsif we just call it repeatedly, and relies on the fact that it does not do anything if VS has been found. Is the subsequent TOOLCHAIN_CHECK_POSSIBLE_VISUAL_STUDIO_ROOT calls failing somehow? * The following code was disabled by you for wsl: WINPATH_BASH="$BASH" BASIC_WINDOWS_REWRITE_AS_WINDOWS_MIXED_PATH([WINPATH_BASH]) Was it reading $BASH that was problematic, or the call to BASIC_WINDOWS_REWRITE_AS_WINDOWS_MIXED_PATH? * The changes in escaping quotes in the env detection script, like: $ECHO "$WINPATH_BASH"' -c "echo VS_PATH=\\\"\"$PATH\"\\\" > set-vs-env.sh"' \ makes me a bit nervous. Have you verified that this does not break on cygwin, or with spaces in the path? I can no longer state exactly why it looks the way it does, but I know that it's the effort of a lot of trial and error on different platforms (cygwin and msys) and different situations. Since we have no good way of running automatic tests for all different platforms, changes such as these makes me nervous. That's not saying that it's bad, but I'd like to see some extra testing. * This looks like some left-over debug output: AC_MSG_NOTICE($VS_ENV_TMP_DIR) And finally some replies to your mail: On 2018-12-09 20:11, Andrew Luo wrote: Hi Everyone, I've been working on getting the OpenJDK to build on WSL (Windows Subsystem for Linux). Currently, our Windows build uses Cygwin. Given that WSL is provided with newer versions of the OS (and doesn't suffer from many of the issues that Cygwin does, given that it is built into the Windows kernel), I think it would be great if OpenJDK would support building on WSL. I've attached a patch with my proposed changes. One important thing to note is that the WSL build targets Windows. It is also possible to use WSL to target itself (a WSL Linux binary) or even other distributions of Linux. I have not implemented that yet, but I think I could do that as a next step if you guys think it would be useful (at least I think it would be useful, then you can test your changes for both Windows and Linux on one system...). I think if you just run configure ordinarily, it will behave like a Linux system and build the Linux image right out-of-the-box..? But then again, maybe that behavior is negated by your changes to config.guess and platform.m4. So maybe we need a flag to configure to control this... Steps in case you want to try this out: 1. Due to autotools not handling spaces well, you have to create symlinks in Windows that will allow you to access Windows Kits and the VC++ compiler without spaces in the path: mklink /D C:\VS "C:\Program Files (x86)\Microsoft Visual Studio" mklink /D C:\WindowsKits "C:\Program Files (x86)\Windows Kits" That's a bit odd. We encounter spaces in paths on Windows normally on
