Yao-MR opened a new issue #5487:
URL: https://github.com/apache/dolphinscheduler/issues/5487
Dolphin scheduler 目前已经移除了数据质量检测，
可见在配置文件中也已经移除了对 相关数据质量涉及的db的
但是代码中依旧存在TaskRecordDao对数据质量的query,
并且SELECT * FROM eamp_hive_log_hd WHERE PROC_NAME='%s' and PROC_DATE like
'%s'"
中涉及的eamp_hive_log_hd db明显已经不存在于配置的默认数据库中，
但是在重要的抽象类AbstractTask 中依旧存在对
TaskRecordDao的数据质量检测逻辑的判定，建议移除来保持对重要抽象类的纯净
public void after() {
if (getExitStatusCode() == Constants.EXIT_CODE_SUCCESS) {
// task recor flat : if true , start up qianfan
if (TaskRecordDao.getTaskRecordFlag()
&&
TaskType.typeIsNormalTask(taskExecutionContext.getTaskType())) {
AbstractParameters params =
TaskParametersUtils.getParameters(taskExecutionContext.getTaskType(),
taskExecutionContext.getTaskParams());
// replace placeholder
Map<String, Property> paramsMap =
ParamUtils.convert(ParamUtils.getUserDefParamsMap(taskExecutionContext.getDefinedParams()),
taskExecutionContext.getDefinedParams(),
params.getLocalParametersMap(),
CommandType.of(taskExecutionContext.getCmdTypeIfComplement()),
taskExecutionContext.getScheduleTime());
if (paramsMap != null && !paramsMap.isEmpty()
&& paramsMap.containsKey("v_proc_date")) {
String vProcDate =
paramsMap.get("v_proc_date").getValue();
if (!StringUtils.isEmpty(vProcDate)) {
TaskRecordStatus taskRecordState =
TaskRecordDao.getTaskRecordState(taskExecutionContext.getTaskName(), vProcDate);
logger.info("task record status : {}",
taskRecordState);
if (taskRecordState == TaskRecordStatus.FAILURE) {
setExitStatusCode(Constants.EXIT_CODE_FAILURE);
}
}
}
}
} else if (getExitStatusCode() == Constants.EXIT_CODE_KILL) {
setExitStatusCode(Constants.EXIT_CODE_KILL);
} else {
setExitStatusCode(Constants.EXIT_CODE_FAILURE);
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
