Re: RFR: JDK-8266396: Add VSCMD_DEBUG for solving WINSDK_DIR build error [v2]
On Mon, 10 May 2021 05:54:51 GMT, John Neffenger wrote: >> The Windows build calls a series of batch files to get the Visual Studio >> paths and environment variables. The batch files are a black box: any >> messages they print are discarded. If anything goes wrong, the only signs >> are a vague Gradle exception and a corrupted properties file. >> >> This has been causing problems for years. There are at least 49 messages on >> the mailing since 2017 that discuss the exception and ways to work around it. >> >> This pull request lets you enable the batch file messages, shedding light on >> their internal workings and allowing you to catch any errors at their >> source. Specifically, it adds the variable `VSCMD_DEBUG` to the environment >> of `genVSproperties.bat` and documents its use. >> >> ### Before >> >> For example, if your `PATH` environment variable is missing >> `C:/Windows/System32`, like mine was, you'll see the following errors: >> >> >> $ gradle sdk >> Dependency verification is an incubating feature. >> >> FAILURE: Build failed with an exception. >> >> * Where: >> Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108 >> >> * What went wrong: >> A problem occurred evaluating script. >>> FAIL: WINSDK_DIR not defined >> >> ︙ >> >> BUILD FAILED in 4s >> >> >> *Me:* What's a `WINSDK_DIR`? The Wiki says, "This means that you will have >> to define these paths manually." I guess this is normal. ️ >> >> ### After >> >> With this change, you can set the debug level to "3" in the `win.gradle` >> file and get a better picture of the problem: >> >> >> $ gradle sdk >> Dependency verification is an incubating feature. >> >>> Configure project : >> 'reg' is not recognized as an internal or external command, >> operable program or batch file. >> 'reg' is not recognized as an internal or external command, >> operable program or batch file. >> 'reg' is not recognized as an internal or external command, >> operable program or batch file. >> >> ︙ >> >> 'powershell.exe' is not recognized as an internal or external command, >> operable program or batch file. >> >> FAILURE: Build failed with an exception. >> >> * Where: >> Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108 >> >> * What went wrong: >> A problem occurred evaluating script. >>> FAIL: WINSDK_DIR not defined >> >> ︙ >> >> BUILD FAILED in 3s >> >> >> *Me:* Oh, it's just a "command not found" error. I'll check my `PATH`. 邏 > > John Neffenger has updated the pull request with a new target base due to a > merge or a rebase. The incremental webrev excludes the unrelated changes > brought in by the merge/rebase. The pull request contains three additional > commits since the last revision: > > - JDK-8266396: Allow VSCMD_DEBUG to work as intended > - Merge branch 'master' into vscmd-debug > - JDK-8266396: Add VSCMD_DEBUG to genVSproperties.bat environment I committed a better proposal, thanks to Kevin's suggestion, and after seeing what it takes to create the `trace.bat` file in my previous comment. Unfortunately, it's not enough to define `VSCMD_DEBUG` externally. The `cmd /q` option (in `win.gradle`) and the redirection of output to `NUL` (in `genVSproperties.bat`) conspire to suppress the requested information, regardless of the variable's value. So this commit adds the minimum changes for `VSCMD_DEBUG` to work. Instead of redirecting output to `NUL`, it redirects output to a new log file called `vcvarsall.log`. Because the trace-level debugging of `VSCMD_DEBUG=3` depends on `echo` being enabled, it adds `echo off` and `echo on` commands selectively in the OpenJFX batch file instead of using the global `cmd /q` option in the Gradle build file. The table below shows the number of debugging lines that are logged for each value of `VSCMD_DEBUG`, both before this pull request (on the `master` branch) and after the latest commit (on the pull request branch): | VSCMD_DEBUG | Level| Before | After | |:---:|:-|---:|--:| | undefined | none | 0 |10 | | 1 | basic| 0 |93 | | 2 | detailed | 0 | 615 | | 3 | trace| 2 | 5,832 | With this change, `VSCMD_DEBUG` actually works as intended. When the build is successful, there's just an extra 10-line log file in the build directory. If something goes wrong, though, developers can set the environment variable and run `gradle help`. The log file will then contain all the information needed to fix the error. If they're having trouble figuring it out, they can post the log file and ask for help on the mailing list. - PR: https://git.openjdk.java.net/jfx/pull/488
Re: RFR: JDK-8266396: Add VSCMD_DEBUG for solving WINSDK_DIR build error [v2]
> The Windows build calls a series of batch files to get the Visual Studio > paths and environment variables. The batch files are a black box: any > messages they print are discarded. If anything goes wrong, the only signs are > a vague Gradle exception and a corrupted properties file. > > This has been causing problems for years. There are at least 49 messages on > the mailing since 2017 that discuss the exception and ways to work around it. > > This pull request lets you enable the batch file messages, shedding light on > their internal workings and allowing you to catch any errors at their source. > Specifically, it adds the variable `VSCMD_DEBUG` to the environment of > `genVSproperties.bat` and documents its use. > > ### Before > > For example, if your `PATH` environment variable is missing > `C:/Windows/System32`, like mine was, you'll see the following errors: > > > $ gradle sdk > Dependency verification is an incubating feature. > > FAILURE: Build failed with an exception. > > * Where: > Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108 > > * What went wrong: > A problem occurred evaluating script. >> FAIL: WINSDK_DIR not defined > > ︙ > > BUILD FAILED in 4s > > > *Me:* What's a `WINSDK_DIR`? The Wiki says, "This means that you will have to > define these paths manually." I guess this is normal. ️ > > ### After > > With this change, you can set the debug level to "3" in the `win.gradle` file > and get a better picture of the problem: > > > $ gradle sdk > Dependency verification is an incubating feature. > >> Configure project : > 'reg' is not recognized as an internal or external command, > operable program or batch file. > 'reg' is not recognized as an internal or external command, > operable program or batch file. > 'reg' is not recognized as an internal or external command, > operable program or batch file. > > ︙ > > 'powershell.exe' is not recognized as an internal or external command, > operable program or batch file. > > FAILURE: Build failed with an exception. > > * Where: > Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108 > > * What went wrong: > A problem occurred evaluating script. >> FAIL: WINSDK_DIR not defined > > ︙ > > BUILD FAILED in 3s > > > *Me:* Oh, it's just a "command not found" error. I'll check my `PATH`. 邏 John Neffenger has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - JDK-8266396: Allow VSCMD_DEBUG to work as intended - Merge branch 'master' into vscmd-debug - JDK-8266396: Add VSCMD_DEBUG to genVSproperties.bat environment - Changes: - all: https://git.openjdk.java.net/jfx/pull/488/files - new: https://git.openjdk.java.net/jfx/pull/488/files/248d0f72..6f6f4932 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jfx=488=01 - incr: https://webrevs.openjdk.java.net/?repo=jfx=488=00-01 Stats: 227 lines in 11 files changed: 179 ins; 21 del; 27 mod Patch: https://git.openjdk.java.net/jfx/pull/488.diff Fetch: git fetch https://git.openjdk.java.net/jfx pull/488/head:pull/488 PR: https://git.openjdk.java.net/jfx/pull/488
Re: RFR: JDK-8266396: Add VSCMD_DEBUG for solving WINSDK_DIR build error
On Fri, 7 May 2021 13:42:49 GMT, Kevin Rushforth wrote:
> It would be more convenient to ask the developer set `VSCMD_DEBUG` via an env
> variable, rather than asking them to edit `win.gradle`.
Thanks, Kevin. That is the most direct approach. I didn't document it that way
for the following reasons:
1. Using the `export` command under Cygwin sets the variable for the *entire*
build. In the current commit, the variable, when defined, is set only for the
duration of the `genVSproperties.bat` batch file and the batch files it calls.
2. I would like to see this documented in the code, even if just as comments,
as opposed to the Wiki. When I got the error, the first thing I did was search
through the code base for `WINSDK_DIR`. I would have found the lines added by
this pull request right away.
Don't let the name confused you. The variable is not a debugging variable; it
doesn't enable debug trace logging. What it actually does is prevent the batch
files from sending both standard output and standard error to "NUL", which they
do by default. Defining it with `VSCMD_DEBUG=3` really means, "Please don't
silently discard really important error messages!" You don't expect all error
messages to be hidden from you when you invoke 3,985 Windows batch file
statements, but that's exactly what's happening now:
$ cloc trace.bat
1 text file.
1 unique file.
0 files ignored.
github.com/AlDanial/cloc v 1.82 T=0.01 s (73.6 files/s, 438775.4 lines/s)
---
Language files blankcomment code
---
DOS Batch1 1938 40 3985
---
I'll up the ante and suggest that we set the value to "3" by default in the
build file (and disable telemetry). Then the batch files will behave as one
would expect. I tested all values of the variable (undefined, valid, and
invalid), and the generated file `windows_tools.properties` was identical in
all cases.
Otherwise, where do you suggest we document this? If you disfavor the current
options ("" and "3"), what do you think of making it three comment lines in the
`win.gradle` file?
-
PR: https://git.openjdk.java.net/jfx/pull/488
Re: RFR: JDK-8266396: Add VSCMD_DEBUG for solving WINSDK_DIR build error
On Thu, 6 May 2021 20:39:11 GMT, John Neffenger wrote: > The Windows build calls a series of batch files to get the Visual Studio > paths and environment variables. The batch files are a black box: any > messages they print are discarded. If anything goes wrong, the only signs are > a vague Gradle exception and a corrupted properties file. > > This has been causing problems for years. There are at least 49 messages on > the mailing since 2017 that discuss the exception and ways to work around it. > > This pull request lets you enable the batch file messages, shedding light on > their internal workings and allowing you to catch any errors at their source. > Specifically, it adds the variable `VSCMD_DEBUG` to the environment of > `genVSproperties.bat` and documents its use. > > ### Before > > For example, if your `PATH` environment variable is missing > `C:/Windows/System32`, like mine was, you'll see the following errors: > > > $ gradle sdk > Dependency verification is an incubating feature. > > FAILURE: Build failed with an exception. > > * Where: > Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108 > > * What went wrong: > A problem occurred evaluating script. >> FAIL: WINSDK_DIR not defined > > ︙ > > BUILD FAILED in 4s > > > *Me:* What's a `WINSDK_DIR`? The Wiki says, "This means that you will have to > define these paths manually." I guess this is normal. ️ > > ### After > > With this change, you can set the debug level to "3" in the `win.gradle` file > and get a better picture of the problem: > > > $ gradle sdk > Dependency verification is an incubating feature. > >> Configure project : > 'reg' is not recognized as an internal or external command, > operable program or batch file. > 'reg' is not recognized as an internal or external command, > operable program or batch file. > 'reg' is not recognized as an internal or external command, > operable program or batch file. > > ︙ > > 'powershell.exe' is not recognized as an internal or external command, > operable program or batch file. > > FAILURE: Build failed with an exception. > > * Where: > Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108 > > * What went wrong: > A problem occurred evaluating script. >> FAIL: WINSDK_DIR not defined > > ︙ > > BUILD FAILED in 3s > > > *Me:* Oh, it's just a "command not found" error. I'll check my `PATH`. 邏 It would be more convenient to ask the developer set `VSCMD_DEBUG` via an env variable, rather than asking them to edit `win.gradle`. That way, the instructions would be: export VSCMD_DEBUG=3 export VSCMD_SKIP_SENDTELEMETRY=1 gradle - PR: https://git.openjdk.java.net/jfx/pull/488
Re: RFR: JDK-8266396: Add VSCMD_DEBUG for solving WINSDK_DIR build error
On 5/6/21 1:46 PM, John Neffenger wrote: $ reg query "HKLM\SOFTWARE\Microsoft\VisualStudio\VSPerf" \ The last part of my comment was truncated by Skara when it tripped over the '/v' Windows command-line option. It's repeated below for the mailing list ... We could instead set the default value to "3". In that case, I get two error messages during the Windows build due to following registry query failing on my system: $ reg query "HKLM\SOFTWARE\Microsoft\VisualStudio\VSPerf" /v "CollectionToolsDir2019" ERROR: The system was unable to find the specified registry key or value. ### Wiki I suggest that we remove the "Missing paths issue" section from the Wiki, added on October 31, 2019. It only led me astray. If you get `WINSDK_DIR not defined`, it's an error, and you should find its source and fix it. This pull request can help.
Re: RFR: JDK-8266396: Add VSCMD_DEBUG for solving WINSDK_DIR build error
On Thu, 6 May 2021 20:39:11 GMT, John Neffenger wrote:
> The Windows build calls a series of batch files to get the Visual Studio
> paths and environment variables. The batch files are a black box: any
> messages they print are discarded. If anything goes wrong, the only signs are
> a vague Gradle exception and a corrupted properties file.
>
> This has been causing problems for years. There are at least 49 messages on
> the mailing since 2017 that discuss the exception and ways to work around it.
>
> This pull request lets you enable the batch file messages, shedding light on
> their internal workings and allowing you to catch any errors at their source.
> Specifically, it adds the variable `VSCMD_DEBUG` to the environment of
> `genVSproperties.bat` and documents its use.
>
> ### Before
>
> For example, if your `PATH` environment variable is missing
> `C:/Windows/System32`, like mine was, you'll see the following errors:
>
>
> $ gradle sdk
> Dependency verification is an incubating feature.
>
> FAILURE: Build failed with an exception.
>
> * Where:
> Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108
>
> * What went wrong:
> A problem occurred evaluating script.
>> FAIL: WINSDK_DIR not defined
>
> ︙
>
> BUILD FAILED in 4s
>
>
> *Me:* What's a `WINSDK_DIR`? The Wiki says, "This means that you will have to
> define these paths manually." I guess this is normal. ️
>
> ### After
>
> With this change, you can set the debug level to "3" in the `win.gradle` file
> and get a better picture of the problem:
>
>
> $ gradle sdk
> Dependency verification is an incubating feature.
>
>> Configure project :
> 'reg' is not recognized as an internal or external command,
> operable program or batch file.
> 'reg' is not recognized as an internal or external command,
> operable program or batch file.
> 'reg' is not recognized as an internal or external command,
> operable program or batch file.
>
> ︙
>
> 'powershell.exe' is not recognized as an internal or external command,
> operable program or batch file.
>
> FAILURE: Build failed with an exception.
>
> * Where:
> Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108
>
> * What went wrong:
> A problem occurred evaluating script.
>> FAIL: WINSDK_DIR not defined
>
> ︙
>
> BUILD FAILED in 3s
>
>
> *Me:* Oh, it's just a "command not found" error. I'll check my `PATH`. 邏
### Notes
The sequence of nested calls to the batch files is shown below, where
`C:/Program Files (x86)/Microsoft Visual Studio` is the root directory of the
Visual Studio files:
buildSrc/genVSproperties.bat
↳ "2019/Community/VC/Auxiliary/Build/vcvarsall.bat"
↳ "2019/Community/Common7/Tools/VsDevCmd.bat"
↳ "2019/Community/Common7/Tools/vsdevcmd/core/winsdk.bat"
The file `VsDevCmd.bat` calls `powershell.exe` to send telemetry to Microsoft
when `VSCMD_DEBUG` is defined. To avoid sending telemetry, define the
environment variable `VSCMD_SKIP_SENDTELEMETRY`. For example, in Cygwin you can
define:
export VSCMD_SKIP_SENDTELEMETRY=1
This pull request sets `VSCMD_DEBUG` to the empty string (""). In batch files,
setting a variable to the empty string is equivalent to leaving the variable
undefined. Furthermore, setting the value in Gradle to the string "3" is the
same as setting the value to the number 3. To verify, I replaced
`genVSproperties.bat` with this batch file:
@echo off
if defined VSCMD_DEBUG (
echo VSCMD_DEBUG is defined 1>&2
) else (
echo VSCMD_DEBUG is not defined 1>&2
)
if "%VSCMD_DEBUG%" NEQ "" (
echo VSCMD_DEBUG = "%VSCMD_DEBUG%" 1>&2
) else (
echo VSCMD_DEBUG is the empty string "%VSCMD_DEBUG%" 1>&2
)
We could instead set the default value to "3". In that case, I get two error
messages during the Windows build due to following registry query failing on my
system:
$ reg query "HKLM\SOFTWARE\Microsoft\VisualStudio\VSPerf" \
-
PR: https://git.openjdk.java.net/jfx/pull/488
RFR: JDK-8266396: Add VSCMD_DEBUG for solving WINSDK_DIR build error
The Windows build calls a series of batch files to get the Visual Studio paths and environment variables. The batch files are a black box: any messages they print are discarded. If anything goes wrong, the only signs are a vague Gradle exception and a corrupted properties file. This has been causing problems for years. There are at least 49 messages on the mailing since 2017 that discuss the exception and ways to work around it. This pull request lets you enable the batch file messages, shedding light on their internal workings and allowing you to catch any errors at their source. Specifically, it adds the variable `VSCMD_DEBUG` to the environment of `genVSproperties.bat` and documents its use. ### Before For example, if your `PATH` environment variable is missing `C:/Windows/System32`, like mine was, you'll see the following errors: $ gradle sdk Dependency verification is an incubating feature. FAILURE: Build failed with an exception. * Where: Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108 * What went wrong: A problem occurred evaluating script. > FAIL: WINSDK_DIR not defined ︙ BUILD FAILED in 4s *Me:* What's a `WINSDK_DIR`? The Wiki says, "This means that you will have to define these paths manually." I guess this is normal. ️ ### After With this change, you can set the debug level to "3" in the `win.gradle` file and get a better picture of the problem: $ gradle sdk Dependency verification is an incubating feature. > Configure project : 'reg' is not recognized as an internal or external command, operable program or batch file. 'reg' is not recognized as an internal or external command, operable program or batch file. 'reg' is not recognized as an internal or external command, operable program or batch file. ︙ 'powershell.exe' is not recognized as an internal or external command, operable program or batch file. FAILURE: Build failed with an exception. * Where: Script 'C:\cygwin64\home\john\src\jfx\buildSrc\win.gradle' line: 108 * What went wrong: A problem occurred evaluating script. > FAIL: WINSDK_DIR not defined ︙ BUILD FAILED in 3s *Me:* Oh, it's just a "command not found" error. I'll check my `PATH`. 邏 - Commit messages: - JDK-8266396: Add VSCMD_DEBUG to genVSproperties.bat environment Changes: https://git.openjdk.java.net/jfx/pull/488/files Webrev: https://webrevs.openjdk.java.net/?repo=jfx=488=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266396 Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jfx/pull/488.diff Fetch: git fetch https://git.openjdk.java.net/jfx pull/488/head:pull/488 PR: https://git.openjdk.java.net/jfx/pull/488
