khandelwal-prateek commented on code in PR #4113:
URL: https://github.com/apache/gobblin/pull/4113#discussion_r2253112881
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/local/LocalTaskStateTracker.java:
##########
@@ -89,8 +90,11 @@ public void onTaskRunCompletion(Task task) {
try {
// Check the task state and handle task retry if task failed and
// it has not reached the maximum number of retries
+
WorkUnitState.WorkingState state = task.getTaskState().getWorkingState();
- if (state == WorkUnitState.WorkingState.FAILED && task.getRetryCount() <
this.maxTaskRetries) {
+ boolean dataQualityEnabled =
jobState.getPropAsBoolean(ConfigurationKeys.DATA_QUALITY_ENABLED_KEY,
ConfigurationKeys.DEFAULT_DATA_QUALITY_ENABLED);
+ String dataQualityResult =
task.getTaskState().getProp(ConfigurationKeys.TASK_LEVEL_POLICY_RESULT_KEY);
+ if (((dataQualityEnabled &&
DataQualityStatus.FAILED.name().equals(dataQualityResult)) || state ==
WorkUnitState.WorkingState.FAILED) && task.getRetryCount() <
this.maxTaskRetries) {
Review Comment:
if `dataQualityEnabled = true` && dataQualityResult is `FAILED`, then task
state would always be `FAILED`.. we don't need to check for DQ result
explicitly for retrying
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
