phet commented on code in PR #4084:
URL: https://github.com/apache/gobblin/pull/4084#discussion_r1884228067
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/DagProcessingEngine.java:
##########
@@ -149,6 +158,16 @@ public void run() {
dagTask.conclude();
log.info(dagProc.contextualizeStatus("concluded dagTask"));
} catch (Exception e) {
+ log.error("DagProcEngineThread: " +
dagProc.contextualizeStatus("error"), e);
+ if (!DagProcessingEngine.isTransientException(e)) {
+ DagActionStore.DagAction dagAction = dagTask.getDagAction();
+ if (dagAction != null) {
+ log.warn(dagProc.contextualizeStatus("ignoring non-transient
exception by concluding so no retries"));
+ }
+
dagManagementStateStore.getDagManagerMetrics().dagProcessingNonRetryableExceptionMeter.mark();
+ dagTask.conclude();
+ }
+ // TODO add the else block for transient exceptions and add conclude
task only if retry limit is not breached
log.error("DagProcEngineThread: " +
dagProc.contextualizeStatus("error"), e);
Review Comment:
actually seems you did reorder... but this is actually a dup. keep only the
first and also move this up w/ it, before the special handling:
```
dagManagementStateStore.getDagManagerMetrics().dagProcessingExceptionMeter.mark();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
