Hisoka-X commented on code in PR #7693:
URL: https://github.com/apache/seatunnel/pull/7693#discussion_r1767835724
##########
seatunnel-engine/seatunnel-engine-core/src/main/java/org/apache/seatunnel/engine/core/job/JobStatus.java:
##########
@@ -38,6 +38,9 @@ public enum JobStatus {
/** The job is already running, and each pipeline is already running. */
RUNNING(EndState.NOT_END),
+ /** The job is waiting for resources. */
+ PENDING(EndState.NOT_END),
Review Comment:
Please move `PENDING` after `CREATED`.
##########
docs/en/seatunnel-engine/hybrid-cluster-deployment.md:
##########
@@ -136,6 +136,22 @@ seatunnel:
classloader-cache-mode: true
```
+### 4.6 Job Scheduling Strategy
+
+When resources are insufficient, the job scheduling strategy can be configured
in the following two modes:
+
+1. `WAIT`: Wait for resources to be available.
+
+2. `REJECT`: Reject the job, default value.
+
+Example
+
+```yaml
+seatunnel:
+engine:
+job-schedule-strategy: WAIT
Review Comment:
```suggestion
```yaml
seatunnel:
engine:
job-schedule-strategy: WAIT
```
##########
seatunnel-engine/seatunnel-engine-common/src/main/java/org/apache/seatunnel/engine/common/config/server/ServerConfigOptions.java:
##########
@@ -138,6 +138,13 @@ public class ServerConfigOptions {
.defaultValue(1440)
.withDescription("The expire time of history jobs.time
unit minute");
+ public static final Option<ScheduleStrategy> JOB_SCHEDULE_STRATEGY =
+ Options.key("job-schedule-strategy")
+ .type(new TypeReference<ScheduleStrategy>() {})
Review Comment:
```suggestion
.enumType(ScheduleStrategy.class)
```
##########
seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/CoordinatorService.java:
##########
@@ -195,6 +212,63 @@ public CoordinatorService(
masterActiveListener = Executors.newSingleThreadScheduledExecutor();
masterActiveListener.scheduleAtFixedRate(
this::checkNewActiveMaster, 0, 100, TimeUnit.MILLISECONDS);
+ isJobPending =
engineConfig.getScheduleStrategy().equals(ScheduleStrategy.WAIT);
+ if (isJobPending) {
+ ScheduledExecutorService scheduler =
Executors.newScheduledThreadPool(1);
+ scheduler.scheduleAtFixedRate(this::pendingJobSchedule, 0, 10,
TimeUnit.SECONDS);
Review Comment:
So we can only schedule one job in each 10s?
##########
seatunnel-engine/seatunnel-engine-client/src/main/java/org/apache/seatunnel/engine/client/job/JobStatusRunner.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.engine.client.job;
+
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class JobStatusRunner implements Runnable {
+ private final JobClient jobClient;
+ private final Long jobId;
+
+ public JobStatusRunner(JobClient jobClient, Long jobId) {
+ this.jobClient = jobClient;
+ this.jobId = jobId;
+ }
+
+ @Override
+ public void run() {
+ Thread.currentThread().setName("job-status-runner-" + jobId);
+ try {
+ String jobStatus = jobClient.getJobStatus(jobId);
Review Comment:
I think we should only print job status is `PENDING`, and when job status
turn from `PENDING` to next stage.
##########
docs/en/seatunnel-engine/separated-cluster-deployment.md:
##########
@@ -280,6 +280,22 @@ netty-common-4.1.89.Final.jar
seatunnel-hadoop3-3.1.4-uber.jar
```
+### 4.7 Job Scheduling Strategy
+
+When resources are insufficient, the job scheduling strategy can be configured
in the following two modes:
+
+1. `WAIT`: Wait for resources to be available.
+
+2. `REJECT`: Reject the job, default value.
+
+Example
+
+```yaml
+seatunnel:
+engine:
+job-schedule-strategy: WAIT
Review Comment:
```suggestion
seatunnel:
engine:
job-schedule-strategy: WAIT
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
