jerryshao commented on code in PR #13132:
URL: https://github.com/apache/gravitino/pull/13132#discussion_r4002867632
##########
core/src/test/java/org/apache/gravitino/job/local/TestLocalJobExecutor.java:
##########
@@ -159,6 +160,41 @@ public void testSubmitJobFailure() throws IOException {
Assertions.assertEquals(JobHandle.Status.FAILED,
jobExecutor.getJobStatus(jobId));
}
+ @Test
+ public void testSubmitSparkJobRejectedWhenSparkSubmitIsNotAvailable() throws
IOException {
+ File sparkHome = new File(workingDir, "spark");
+ LocalJobExecutor exec = new LocalJobExecutor();
+ exec.initialize(
+ ImmutableMap.of(LocalJobExecutorConfigs.SPARK_HOME,
sparkHome.getAbsolutePath()));
+
+ try {
+ SparkJobTemplate template =
+ SparkJobTemplate.builder()
+ .withName("spark-job")
+ .withExecutable(new File(workingDir,
"spark-demo.jar").getAbsolutePath())
+ .withClassName("com.example.MainClass")
+ .build();
+
+ // spark-submit does not exist, the job is rejected at submission
instead of being queued.
+ IllegalArgumentException e =
+ Assertions.assertThrows(IllegalArgumentException.class, () ->
exec.submitJob(template));
+ Assertions.assertTrue(e.getMessage().contains("spark-submit is not found
or not executable"));
+
+ // Once spark-submit is available, the same job is accepted.
+ File sparkSubmit = new File(sparkHome, "bin/spark-submit");
+ FileUtils.writeStringToFile(sparkSubmit, "#!/bin/sh\nexit 0\n", "UTF-8");
Review Comment:
`FileUtils.writeStringToFile` creates missing parent directories: it opens
the file through `FileUtils.openOutputStream`, which calls `mkdirs()` on the
parent. So `bin/` does not need to be created first, and this test passes as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
