Blazer-007 commented on code in PR #4135:
URL: https://github.com/apache/gobblin/pull/4135#discussion_r2296983135
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/Task.java:
##########
@@ -329,17 +331,37 @@ private void computeAndUpdateTaskDataQuality() {
try {
if (result != null) {
forkDataQualityStatus = DataQualityStatus.valueOf(result);
+ } else {
+ forkDataQualityStatus = DataQualityStatus.NOT_EVALUATED;
}
} catch (IllegalArgumentException e) {
Log.warn("Unknown data quality status encountered: " + result);
forkDataQualityStatus = DataQualityStatus.UNKNOWN;
}
- if (DataQualityStatus.FAILED == forkDataQualityStatus) {
- overallTaskDataQuality = DataQualityStatus.FAILED;
+ /*
+ * If any fork fails, overall status should be FAILED
+ * FAILED status cannot be overridden by subsequent successes
+ * Handle NOT_EVALUATED and UNKNOWN appropriately
+ * If forkDataQualityStatus is PASSED and overall is not
FAILED/UNKNOWN, keep PASSED
+ */
Review Comment:
Are we handling this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
