demery-pivotal opened a new pull request #6280:
URL: https://github.com/apache/geode/pull/6280
Updated Geode's build system to use Gradle 6.8.3:
- Updated the gradle wrapper to use Gradle 6.8.3.
- Updated geode-assembly/build.gradle to make the ivy "repository"
compatible with Gradle 6.8.3.
- Changed ./gradle.properties to specify 'minimumGradleVersion=6.8'.
- Updated the expected-pom.xml file to the format that Gradle 6.8.3
produces.
- Updated all Gradle plugins to the latest versions.
- In buildSrc/build.gradle, added groovy source sets for testing, to
allow running tests of plugin code.
- Updated common "test isolation" code to be compatible with Gradle
6.8.3. See below for details.
- Updated the RepeatTest task to be compatible with Gradle 6.8.3. See
below for details.
- Rewrote the Dockerized test plugin to be compatible with Gradle 6.8.3.
See below for details.
- Added a new 'dunitDockerJVM' project property to allow developers run
tests in Docker on macOS. To do this, use -PtestJVM to identify the
mac's JVM, and use -PdunitDockerJVM to specify the JVM for the test
worker process running in Docker.
- Updated the BUILDING.md file for Gradle 6.8.3. As part of this I added
two versions of the Apache copyright notice to the etc dir: A plain
text version and an xml version that IntelliJ IDEA can import into its
copyright settings.
===== Test isolation code =====
Some code for isolating tests is used by the RepeatTest task and the
Dockerized test plugin. This code was updated to be compatible with
Gradle v6.8.3, and to somewhat reduce its dependence on Gradle internal
implementation details:
- WorkingDirectoryIsolator adjusts a given ProcessBuilder to give it a
unique working directory.
- AdjustableProcessLauncher applies a given Consumer (such as the
isolator, above) to adjust each ProcessBuilder before launching the
process.
- LauncherProxyWorkerProcessBuilder and
LauncherProcessWorkerProcessFactory forcefully update each Gradle exec
handle to use a given ProcessLauncher to launch test worker processes.
- Executers and Workers offer convenience methods for constructing
TestExecuters and ProcessWorker builders and factories.
===== RepeatTest task update =====
Geode's RepeatTest task uses a custom TestExecuter that was copied from
Gradle 5.5 (or earlier) and modified. This custom class is now updated
to be compatible with Gradle 6.8.3, and renamed from
OverriddenTestExecutor to RepeatableTestExecuter.
===== Dockerized test plugin rewrite =====
Removed the modified copy of pedjak's Dockerized test plugin.
Configuration
-------------
- Rewrote gradle/docker.gradle and renamed it as
gradle/multi-process-test.gradle.
- Moved the code that translates project properties into configuration
settings from the plugin to the config class. Renamed the config class
as DockerTestWorkerConfig.
- Moved code that configures command lines and environments for Docker
test workers based on project properties. Thise code was defiend as a
closure in a build script. It is now a method in
DockerTestWorkerConfig.
- Changed command line options that CI scripts pass to Gradle to run
tests in Docker:
- Specify the --max-workers Gradle option to configure the maximum
number of workers.
- Specify the 'testMaxParallelForks' project property to configure
the maximum number of forks for each test task.
- Remove the 'dunitParallelForks' option.
Implementation
--------------
The Dockerized test plugin uses these classes to launches processes in
Docker containers:
- DockerProcess overrides java.lang.Process to represent a test worker
process running in Docker.
- DockerProcessLauncher launches each test worker process in a Docker
container, and creates a DockerProcess to represent and manage it.
- Other classes described in the "Test isolation code" section, above.
The plugin overrides several of Gradle's internal classes to allow test
worker processes running in Docker containers to communicate with
Gradle:
- DockerConnectionAcceptor implements Gradle's ConnectionAcceptor, which
(among other responsibilities), produces a "multi-choice address" that
a test worker process can use to connect to Gradle's messaging server.
Gradle's default implementation produces multi-choice addresses that
processes in Docker containers cannot use.
- DockerMessagingServer implements Gradle's MessagingServer using
DockerConnectionAcceotpr.
- WildcardBindingInetAddressFactory overrides Gradle's
InetAddressFactory to instruct messaging servers to listen on an
address that processes in Dockerized containers can connect to.
Timeouts
--------
Due to a recent surge of CI problems due to delays in Docker operations,
I rewrote the plugins's timeout mechanism to make it configurable and
much more robust:
- Added a 'dockerTimeout' project property, defaulting to 5 minutes.
Each Docker operation will throw an exception if its duration exceeds
this value.
- Log a warning if any Docker operation takes longer than 1 minute. This
can help us to see how often significant delays occur in CI, while
still allowing tests to proceed.
- These exceptions and warnings identify the project for which test
worker process is being launched. This can help us to see whether
there are project-specific factors that affect the delays.
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced in
the commit message?
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `develop`)?
- [ ] Is your initial contribution a single, squashed commit?
- [ ] Does `gradlew build` run cleanly?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
### Note:
Please ensure that once the PR is submitted, check Concourse for build
issues and
submit an update to your PR as soon as possible. If you need help, please
send an
email to [email protected].
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
