rohangarg commented on a change in pull request #12326:
URL: https://github.com/apache/druid/pull/12326#discussion_r832120068
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/common/task/NoopTask.java
##########
@@ -176,6 +178,14 @@ public static NoopTask withGroupId(String groupId)
return new NoopTask(null, groupId, null, 0, 0, null, null, null);
}
+ public static NoopTask withJavaOptsContext(String javaOpts, List<String>
javaOptsArray)
+ {
+ Map<String, Object> context = new HashMap<>();
Review comment:
this should be `Map<String, Object> context = new
HashMap<>(getContext());` so that we keep the original one as well
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/overlord/ForkingTaskRunner.java
##########
@@ -219,6 +220,23 @@ public TaskStatus call()
);
}
+ // Override task specific javaOptsArray
+ try {
+ List<String> taskJavaOptsArray =
jsonMapper.convertValue(
+
task.getContextValue(ForkingTaskRunnerConfig.JAVA_OPTS_ARRAY_PROPERTY),
+ new TypeReference<List<String>>() {}
+ );
+ if (taskJavaOptsArray != null) {
+ Iterables.addAll(command, taskJavaOptsArray);
+ }
+ }
+ catch (Exception e) {
+ throw new IllegalArgumentException(
+ "javaOptsArray in context of task: " + task + "
must be an array of strings.",
Review comment:
we should replace `javaOptsArray` with
`ForkingTaskRunnerConfig.JAVA_OPTS_ARRAY_PROPERTY`.
Also, I think task doesn't implement a `toString` properly generally -
should be replaced with `taskId`
##########
File path:
indexing-service/src/test/java/org/apache/druid/indexing/overlord/ForkingTaskRunnerTest.java
##########
@@ -353,4 +353,76 @@ int waitForTaskProcessToComplete(Task task, ProcessHolder
processHolder, File lo
Assert.assertEquals(TaskState.FAILED, status.getStatusCode());
Assert.assertEquals("task failure test", status.getErrorMsg());
}
+
+ @Test
+ public void testJavaOptsAndJavaOptsArrayOverride() throws
ExecutionException, InterruptedException
+ {
+ ObjectMapper mapper = new DefaultObjectMapper();
+ String javaOpts = "-Xmx1g -Xms1g";
+ List<String> javaOptsArray = ImmutableList.of("-Xmx10g", "-Xms10g");
+ Task task = NoopTask.withJavaOptsContext(javaOpts, javaOptsArray);
+ ForkingTaskRunner forkingTaskRunner = new ForkingTaskRunner(
+ new ForkingTaskRunnerConfig(),
+ new TaskConfig(
+ null,
+ null,
+ null,
+ null,
+ ImmutableList.of(),
+ false,
+ new Period("PT0S"),
+ new Period("PT10S"),
+ ImmutableList.of(),
+ false,
+ false,
+ TaskConfig.BATCH_PROCESSING_MODE_DEFAULT.name()
+ ),
+ new WorkerConfig(),
+ new Properties(),
+ new NoopTaskLogs(),
+ mapper,
+ new DruidNode("middleManager", "host", false, 8091, null, true, false),
+ new StartupLoggingConfig()
+ )
+ {
+ @Override
+ ProcessHolder runTaskProcess(List<String> command, File logFile,
TaskLocation taskLocation) throws IOException
+ {
+ ProcessHolder processHolder = Mockito.mock(ProcessHolder.class);
+
Mockito.doNothing().when(processHolder).registerWithCloser(ArgumentMatchers.any());
+ Mockito.doNothing().when(processHolder).shutdown();
+
+ int xmxJavaOptsIndex = 0;
+ int xmxJavaOptsArrayIndex = 0;
+ String statusPath = "status.json";
+ for (int i = 0; i < command.size(); i++) {
+ if (command.get(i).endsWith("status.json")) {
+ statusPath = command.get(i);
+ }
+
+ if ("-Xmx1g".equals(command.get(i))) {
+ xmxJavaOptsIndex = i;
+ }
+ if ("-Xmx10g".equals(command.get(i))) {
+ xmxJavaOptsArrayIndex = i;
+ }
+ }
+ if (0 < xmxJavaOptsIndex && xmxJavaOptsIndex < xmxJavaOptsArrayIndex) {
+ mapper.writeValue(new File(statusPath),
TaskStatus.failure(task.getId(), "javaOpts and javaOptsArray context override
work as expected"));
+ }
+
+ return processHolder;
+ }
+
+ @Override
+ int waitForTaskProcessToComplete(Task task, ProcessHolder processHolder,
File logFile, File reportsFile)
+ {
+ return 0;
+ }
+ };
+
+ final TaskStatus status = forkingTaskRunner.run(task).get();
+ Assert.assertEquals(TaskState.FAILED, status.getStatusCode());
Review comment:
you can fail the task if the assertion is not met - and assert SUCCESS
task run here. the error message can contain details about the assertion failure
that seems more natural
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
