khandelwal-prateek commented on code in PR #4113:
URL: https://github.com/apache/gobblin/pull/4113#discussion_r2251957581
##########
gobblin-core/src/main/java/org/apache/gobblin/qualitychecker/task/TaskLevelPolicyChecker.java:
##########
@@ -28,20 +29,13 @@
* executes each one, and then stores the output
* in a PolicyCheckResults object
*/
+@Getter
public class TaskLevelPolicyChecker {
- /**
- * An enumeration for possible statuses for Data quality checks,
- * its values will be PASSED, FAILED, in case if data quality check
- * evaluation is not performed for Job, it will be NOT_EVALUATED
- */
- public enum DataQualityStatus {
- PASSED,
- FAILED,
- NOT_EVALUATED
- }
private final List<TaskLevelPolicy> list;
private static final Logger LOG =
LoggerFactory.getLogger(TaskLevelPolicyChecker.class);
+ public static final String TASK_LEVEL_POLICY_RESULT_KEY =
"gobblin.task.level.policy.result";
Review Comment:
As discussed, the code supports marking policies as OPTIONAL. In
`jdbc-to-hdfs.template`, the policies are already defined with
`qualitychecker.task.policy.types=OPTIONAL`. Because policy results are
currently maintained in a map that stores only a single value for each failure
type, this would suppress data quality failures in such cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
