xintongsong commented on a change in pull request #13347:
URL: https://github.com/apache/flink/pull/13347#discussion_r486032936
##########
File path:
flink-yarn/src/main/java/org/apache/flink/yarn/WorkerSpecContainerResourceAdapter.java
##########
@@ -143,8 +149,9 @@ private InternalContainerResource
createAndMapContainerResource(final WorkerReso
/**
* Normalize to the minimum integer that is greater or equal to 'value'
and is positive integer multiple of 'unitValue'.
Review comment:
JavaDoc needs update.
##########
File path: flink-yarn/src/test/java/org/apache/flink/yarn/UtilsTest.java
##########
@@ -58,4 +61,30 @@ public void testDeleteApplicationFiles() throws Exception {
assertThat(files.count(), equalTo(0L));
}
}
+
+ @Test
+ public void testGetUnitResource() {
+ YarnConfiguration yarnConfig = new YarnConfiguration();
+
+ yarnConfig.set("yarn.resourcemanager.scheduler.class",
"org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler");
Review comment:
We should always try to avoid magic string literals if possible.
##########
File path:
flink-yarn/src/test/java/org/apache/flink/yarn/WorkerSpecContainerResourceAdapterTest.java
##########
@@ -106,6 +110,31 @@ public void testMatchVcores() {
assertThat(adapter.getWorkerSpecs(containerResource1,
strategy), containsInAnyOrder(workerSpec1, workerSpec2));
assertThat(adapter.getWorkerSpecs(containerResource2,
strategy), contains(workerSpec3));
assertThat(adapter.getWorkerSpecs(containerResource3,
strategy), contains(workerSpec4));
+
+ final int unitMemMB1 = 150;
+ final int unitVcore1 = 15;
+ final WorkerSpecContainerResourceAdapter adapter1 =
+ new WorkerSpecContainerResourceAdapter(
+ getConfigProcessSpecEqualsWorkerSpec(),
+ minMemMB,
+ minVcore,
+ Integer.MAX_VALUE,
+ Integer.MAX_VALUE,
+ unitMemMB1,
+ unitVcore1,
+ Collections.emptyMap());
+
+ final Resource containerResource4 = Resource.newInstance(150,
15);
+
+ assertThat(adapter1.getWorkerSpecs(containerResource1,
strategy), empty());
+ assertThat(adapter1.getWorkerSpecs(containerResource2,
strategy), empty());
+
+
assertThat(adapter1.tryComputeContainerResource(workerSpec1).get(),
is(containerResource4));
+
assertThat(adapter1.tryComputeContainerResource(workerSpec2).get(),
is(containerResource4));
+
assertThat(adapter1.tryComputeContainerResource(workerSpec3).get(),
is(containerResource4));
+
assertThat(adapter1.tryComputeContainerResource(workerSpec4).get(),
is(containerResource4));
+
+ assertThat(adapter1.getWorkerSpecs(containerResource4,
strategy), containsInAnyOrder(workerSpec1, workerSpec2, workerSpec3,
workerSpec4));
Review comment:
I'm not sure whether it is necessary to introduce another adapter.
Should be good enough to tune the parameters of existing cases.
I guess the problem is that the original test case did not make its
intention explicit.
##########
File path:
flink-yarn/src/test/java/org/apache/flink/yarn/WorkerSpecContainerResourceAdapterTest.java
##########
@@ -169,6 +202,31 @@ public void testIgnoreVcores() {
assertThat(adapter.getWorkerSpecs(containerResource4,
strategy), containsInAnyOrder(workerSpec1, workerSpec2, workerSpec3));
assertThat(adapter.getWorkerSpecs(containerResource5,
strategy), contains(workerSpec4));
+
+ final int unitMemMB1 = 150;
+ final int unitVcore1 = 15;
+ final WorkerSpecContainerResourceAdapter adapter1 =
+ new WorkerSpecContainerResourceAdapter(
+ getConfigProcessSpecEqualsWorkerSpec(),
+ minMemMB,
+ minVcore,
+ Integer.MAX_VALUE,
+ Integer.MAX_VALUE,
+ unitMemMB1,
+ unitVcore1,
+ Collections.emptyMap());
+
+ final Resource containerResource6 = Resource.newInstance(150,
15);
+
+
assertThat(adapter1.tryComputeContainerResource(workerSpec1).get(),
is(containerResource6));
+
assertThat(adapter1.tryComputeContainerResource(workerSpec2).get(),
is(containerResource6));
+
assertThat(adapter1.tryComputeContainerResource(workerSpec3).get(),
is(containerResource6));
+
assertThat(adapter1.tryComputeContainerResource(workerSpec4).get(),
is(containerResource6));
+
+
assertThat(adapter1.getEquivalentContainerResource(containerResource4,
strategy), empty());
+
assertThat(adapter1.getEquivalentContainerResource(containerResource5,
strategy), contains(containerResource6));
+
+ assertThat(adapter1.getWorkerSpecs(containerResource3,
strategy), containsInAnyOrder(workerSpec1, workerSpec2, workerSpec3,
workerSpec4));
Review comment:
Same here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
