zixi0825 opened a new issue #4829:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4829
- Function
DataQualityApplication uses spark as execution engine and parse the input
parameter to generate Connector、Executor and Writer .Read data from different
data source through different Connector and use the sql in executor to operate
data. Finally write the result into storage engine through the Writer
- Architecture
- Parameter design
Parse input parameter to generate ConnectorParamters、ExecutorParameters and
WriterParameters
- Flow and DataQualityTask
- DataQualityTask is a interface, all the connectors、executors and
writers must implement the DataQualityTask
- A Flow consist of a list of DataQualityTask
- The DataQualityApplication is to execute all DataQualityTask in
Flow to complete the execution logic
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
