kl0u commented on a change in pull request #14275:
URL: https://github.com/apache/flink/pull/14275#discussion_r535182027
##########
File path:
flink-end-to-end-tests/flink-end-to-end-tests-common/src/main/java/org/apache/flink/tests/util/flink/FlinkContainer.java
##########
@@ -256,31 +258,83 @@ public FlinkContainer build() throws IOException {
.mapToObj(i -> "localhost")
.collect(Collectors.toList()));
- ImageFromDockerfile image = new
ImageFromDockerfile("flink-dist", true)
- .withDockerfileFromBuilder(
- builder -> {
- builder.from("openjdk:"
+ getJavaVersionSuffix())
-
.copy(flinkDistName, "flink")
-
.copy(flinkDistName + "/conf/workers", "workers")
- .cmd(FLINK_BIN
+ "/start-cluster.sh && tail -f /dev/null")
- .build();
- }
- )
- .withFileFromPath("workers",
workersFile)
- .withFileFromPath(flinkDistName,
flinkDist);
+ // Building the docker image is split into two
stages:
+ // 1. build a base image with an immutable
flink-dist
+ // 2. based on the base image add any mutable
files such as e.g. workers files
+ //
+ // This lets us save some time for archiving
and copying big, immutable files
+ // between tests runs.
+ String baseImage = buildBaseImage(flinkDist,
flinkDistName);
+ ImageFromDockerfile configuredImage =
buildConfiguredImage(
+ flinkDistName,
+ workersFile,
+ baseImage);
Optional<Path> logBackupDirectory =
DISTRIBUTION_LOG_BACKUP_DIRECTORY.get();
if (!logBackupDirectory.isPresent()) {
LOG.warn(
"Property {} not set, logs will
not be backed up in case of test failures.",
DISTRIBUTION_LOG_BACKUP_DIRECTORY.getPropertyName());
}
- return new FlinkContainer(image,
numTaskManagers, logBackupDirectory.orElse(null));
+ return new FlinkContainer(
+ configuredImage,
+ numTaskManagers,
+ logBackupDirectory.orElse(null));
+ } catch (Exception e) {
+ throw new RuntimeException("Could not build the
flink-dist image", e);
} finally {
temporaryFolder.delete();
}
}
+ private ImageFromDockerfile buildConfiguredImage(
+ String flinkDistName,
+ Path workersFile,
+ String baseImage) {
+ return new ImageFromDockerfile(
+ "flink-dist-configured")
+ .withDockerfileFromBuilder(
+ builder -> builder.from(baseImage)
+ .copy("workers", flinkDistName
+ "/conf/workers")
Review comment:
I think the arguments here should be reversed:
```
.copy(flinkDistName + "/conf/workers", "workers")
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
