MartijnVisser commented on code in PR #29399:
URL: https://github.com/apache/flink/pull/29399#discussion_r4204112692
##########
flink-runtime/src/test/java/org/apache/flink/runtime/rest/handler/job/JobVertexBackPressureHandlerTest.java:
##########
@@ -427,4 +434,108 @@ public long getLastUpdateTime() {
.collect(Collectors.toList()))
.containsExactly(0, 1);
}
+
+ /**
+ * The handler must not throw when a subtask's attempts are pruned
concurrently (by {@code
+ * SubtaskMetricStore.retainAttempts()}) between its {@code size()}/{@code
containsKey()} checks
+ * and the {@code keySet().iterator().next()} read.
+ */
+ @Test
+ void testGetBackPressureWhenAttemptsRemovedConcurrently() throws Exception
{
+ MetricStore multipleAttemptsMetricStore = new MetricStore();
+ for (MetricDump metricDump : getMultipleAttemptsMetricDumps()) {
+ multipleAttemptsMetricStore.add(metricDump);
+ }
+
+ // Empty subtask 0's attempts in the window between the handler's
check and its iteration.
+ SubtaskMetricStore subtask =
+ multipleAttemptsMetricStore
+ .getTaskMetricStore(
+
TEST_JOB_ID_BACK_PRESSURE_STATS_AVAILABLE.toString(),
+ TEST_JOB_VERTEX_ID.toString())
+ .getAllSubtaskMetricStores()
+ .get(0);
+ Field attemptsField =
SubtaskMetricStore.class.getDeclaredField("attempts");
+ attemptsField.setAccessible(true);
+
+ Map<Integer, ComponentMetricStore> originalAttempts =
+ (Map<Integer, ComponentMetricStore>)
attemptsField.get(subtask);
Review Comment:
The cast is still here, so javac now warns about unchecked operations in
this file. Passing `subtask.getAllAttemptsMetricStores()` to the constructor
avoids both.
##########
flink-runtime/src/test/java/org/apache/flink/runtime/rest/handler/job/JobVertexBackPressureHandlerTest.java:
##########
@@ -427,4 +434,108 @@ public long getLastUpdateTime() {
.collect(Collectors.toList()))
.containsExactly(0, 1);
}
+
+ /**
+ * The handler must not throw when a subtask's attempts are pruned
concurrently (by {@code
+ * SubtaskMetricStore.retainAttempts()}) between its {@code size()}/{@code
containsKey()} checks
+ * and the {@code keySet().iterator().next()} read.
+ */
+ @Test
+ void testGetBackPressureWhenAttemptsRemovedConcurrently() throws Exception
{
+ MetricStore multipleAttemptsMetricStore = new MetricStore();
+ for (MetricDump metricDump : getMultipleAttemptsMetricDumps()) {
+ multipleAttemptsMetricStore.add(metricDump);
+ }
+
+ // Empty subtask 0's attempts in the window between the handler's
check and its iteration.
+ SubtaskMetricStore subtask =
+ multipleAttemptsMetricStore
+ .getTaskMetricStore(
+
TEST_JOB_ID_BACK_PRESSURE_STATS_AVAILABLE.toString(),
+ TEST_JOB_VERTEX_ID.toString())
+ .getAllSubtaskMetricStores()
+ .get(0);
+ Field attemptsField =
SubtaskMetricStore.class.getDeclaredField("attempts");
+ attemptsField.setAccessible(true);
+
+ Map<Integer, ComponentMetricStore> originalAttempts =
+ (Map<Integer, ComponentMetricStore>)
attemptsField.get(subtask);
+ attemptsField.set(subtask, new SelfEmptyingOnKeySet(originalAttempts));
+
+ JobVertexBackPressureHandler handler =
+ new JobVertexBackPressureHandler(
+ () ->
CompletableFuture.completedFuture(restfulGateway),
+ Duration.ofSeconds(10),
+ Map.of(),
+ JobVertexBackPressureHeaders.getInstance(),
+ new MetricFetcher() {
+ @Override
+ public MetricStore getMetricStore() {
+ return multipleAttemptsMetricStore;
+ }
+
+ @Override
+ public void update() {}
+
+ @Override
+ public long getLastUpdateTime() {
+ return 0;
+ }
+ });
+
+ Map<String, String> pathParameters = new HashMap<>();
+ pathParameters.put(
+ JobIDPathParameter.KEY,
TEST_JOB_ID_BACK_PRESSURE_STATS_AVAILABLE.toString());
+ pathParameters.put(JobVertexIdPathParameter.KEY,
TEST_JOB_VERTEX_ID.toString());
+ HandlerRequest<EmptyRequestBody> request =
+ HandlerRequest.resolveParametersAndCreate(
+ EmptyRequestBody.getInstance(),
+ new JobVertexMessageParameters(),
+ pathParameters,
+ Map.of(),
+ List.of());
+
+ assertThatCode(() -> handler.handleRequest(request, restfulGateway))
+ .doesNotThrowAnyException();
+ }
+
+ private static final class SelfEmptyingOnKeySet
Review Comment:
Nit: it empties on more than `keySet()` now, maybe `SelfEmptyingAttempts`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
