Sergey's suggestion to force the elimination of shell tests may not be
far off the mark. :-)
It is getting insanely complicated to support maintain these shell
tests. Not impossible, but very, very fragile.
That being said, I have a small demo test suite that mimics the options
1-4 that I enumerated before, and I have an updated version of jtreg to
run these tests.
This version of jtreg supports the following:
* bin/jtreg has been updated to work correctly on WSL
* new option -cygwin, as well as recent new -wsl option, to force the
use of Cygwin, but neither option is necessary in normal use: jtreg
will autodetect whether to use Cygwin or WSL (or MKS, untested)
* The following combinations are supported on Windows:
* Cygwin, jtreg running on a Windows JDK, test JDK is a Windows JDK.
* WSL, jtreg running on a Windows JDK, test JDK is a Windows JDK.
* WSL, jtreg running on a Linux JDK, test JDK is a Linux JDK.
* jtreg will /not/ support mixed JDKs ... running on one kind of JDK,
and the test JDK is the other kind.
Part of the reason this is absurdly fragile is because of the need to
use "wslpath" to convert command line args, which (I'm presuming) we
don't want to put in our shell scripts. WSLEnv only does some of the
work; it only converts environment variables; it does nothing to help
fix up command lines. Therefore the only way to get shell scripts to
work is for jtreg to "guess" whether an environment variable is going to
be used by WSL (e.g. a path to invoke the test JDK) or whether the
environment variable is going to be used by a Windows binary invoked
from WSL (e.g. a value for a classpath option.)
It is worth noting that we don't have this problem on Cygwin, because
Cygwin will tolerate the use of Windows-style paths, and so
consistent-looking environment variables can be used within the script
and passed to programs. That's not true in the WSL world at this point.
I need to do more testing on the new version of jtreg; I'll push changes
next week.
-- Jon
On 01/26/2019 08:40 AM, Jonathan Gibbons wrote:
I don't think we should force the elimination of shell tests, but we
should definitely encourage it. (i.e. Option 1.)
The underlying theme in options 1-4 is to minimize the effort required
to accommodate the changes needed to support the use of WSL to run
tests; not to impose effort. Carrots, not sticks.
Of course, Sergey, if you'd like to volunteer to convert all the
client tests, that would be your prerogative. Just yesterday, I had a
chat with Phil, giving anecdotes about how we converted almost all of
the langtools shell tests, by writing a library that provided methods
based on the shell commands that we saw in our shell scripts: cp, mv,
rm, diff, grep, etc. Others have done the same for some of the
core-libs tests. We can start a separate thread if you'd like to
discuss such techniques.
-- Jon
On 1/25/19 5:16 PM, Sergey Bylokhov wrote:
No my point was radical drop of all such tests.
On 25/01/2019 17:05, Andrew Luo wrote:
Isn't that option 1?
Thanks,
-Andrew
-----Original Message-----
From: quality-discuss <quality-discuss-boun...@openjdk.java.net> On
Behalf Of Sergey Bylokhov
Sent: Friday, January 25, 2019 5:00 PM
To: Jonathan Gibbons <jonathan.gibb...@oracle.com>;
quality-discuss@openjdk.java.net
Subject: Re: jtreg shell tests
There is one more Option 5.
Drop shell tests from the workspace and provide some examples on how
to write such logic using java api.
On 25/01/2019 16:43, Jonathan Gibbons wrote:
With all the recent discussion regarding how to support the use of
Windows Subsystem for Linux (WSL) as an alternate to Cygwin, it
seems worth writing up some recommendations on writing jtreg shell
tests.
The intent of these notes is that they will evolve into a page in
the jtreg section on the OpenJDK website.
The focus is specifically about different approaches to providing the
ability to run a shell test on all supported platforms, by means of
abstracting the significant differences into a series of
environment variables that are set according to the environment in
which the test is running.
Option 1.
Convert the test to Java. In general, this continues to be the
recommended alternative.
Option 2.
Use a shell `case` statement, like the following, or a variant
thereof:
OS=`uname -s`;
case "$OS" in
Windows* | CYGWIN* )
FS="\\"
PS=";"
NULL=NUL
;;
Linux )
if [ -r $TESTJAVA/bin/java.exe ]; then
FS="\\"
PS=";"
EXE_SUFFIX=".exe"
else
FS="/"
PS=":"
fi
NULL=/dev/null
;;
* )
FS="/"
PS=":"
NULL=/dev/null
esac
Option 3.
Use a shared library script to embody the behavior in the previous
example. jtreg now provides a new `TESTROOT` environment variable,
which makes it easy to reference a shared script in a constant
manner from any shell test, wherever the test is within the test
suite. Since the library script is used to set environment
variables like `FS`, `PS`, and `NULL`, it should be executed with
`source` and not `bash` or `sh`.
Option 4.
jtreg now sets the following environment variables when running a
shell script: `FS`, `PS`, `NULL` and `EXE_SUFFIX`. This may be
enough to completely avoid the need for a `case` statement in each
shell script or the use of a shared library script to set these
variables.
Running scripts standalone.
One concern when working with shell tests has been the ability to
run the test "stand-alone", without the use of jtreg. In the past,
this was seen as justification for the explicit use of the `case`
statement in each shell test. However, the need to run shell tests
standalone no longer seems to be a significant concern. For those
that do want to run shell tests by themselves, it is worth noting
that once a test has been run by jtreg, the ".jtr" file contains
"rerun" sections with details on how to run each action of the
test. You can either copy/paste/edit from the ".jtr" file directly,
or use the jtreg `-show:rerun` option to output the information to
the standard output stream.
--
Best regards, Sergey.
