Hey guys,

With dolphinscheduler landing on ziroom, we have experienced nearly a year
of use and secondary development, summarizing many interesting experiences.
1.dolphinscheduler main use scenario: offline number warehouse field
2.dolphinscheduler's webui provides to create DAG by dragging and dropping,
which is not friendly in the face of business scenarios with complex
correlations.
3.There will be some unique problems for the offline data warehouse
vertical. We developed a more suitable development platform for offline
data warehouse based on dolphinscheduler, a scheduling engine.
4.Based on the above, we have developed a new platform to achieve: complex
Hql automated generation scheduling, data dependency support, more powerful
periodic replenishment, more convenient alerts, clearer log display, and
realized the support of data pedigree and data insight. Currently, we are
also developing the ability to generate and schedule FlinkSQL pedigree
analysis with one click

5.The newly developed platform is designed at the beginning to be
compatible with multiple versions, so it is completely decoupled from
dolphinscheduler, only the packaged jar, put into the lib directory of
dolphinscheduler, and it is done ready.

6.I would like to contribute it as a sub-project of dolphinscheduler,
positioning: the offline data warehouse vertical, to achieve a common
offline data warehouse solution in the industry, to meet the needs of users
to build offline data warehouses.
Related links:https://github.com/apache/dolphinscheduler/issues/12971

Reply via email to