umustafi commented on code in PR #3656:
URL: https://github.com/apache/gobblin/pull/3656#discussion_r1130146197
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/scheduler/GobblinServiceJobScheduler.java:
##########
@@ -290,19 +345,30 @@ private void scheduleSpecsFromCatalog() {
while (batchOfSpecsIterator.hasNext()) {
Spec spec = batchOfSpecsIterator.next();
- try {
- addSpecHelperMethod(spec);
- urisLeftToSchedule.remove(spec.getUri());
- } catch (Exception e) {
- // If there is an uncaught error thrown during compilation, log it
and continue adding flows
- _log.error("Could not schedule spec {} from flowCatalog due to ",
spec, e);
+ FlowSpec flowSpec = (FlowSpec) spec;
+ String cronExpression =
flowSpec.getConfig().getString(ConfigurationKeys.JOB_SCHEDULE_KEY);
+ // Empty string cron expressions should be scheduled by default
+ if (isNextRunWithinRangeToSchedule(cronExpression,
this.thresholdToSkipSchedulingFlowsAfter)) {
Review Comment:
yea that's a good call, if we aren't able to schedule we should remove from
that list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
