Hey guys, With dolphinscheduler landing on ziroom, we have experienced nearly a year of use and secondary development, summarizing many interesting experiences. 1.dolphinscheduler main use scenario: offline number warehouse field 2.dolphinscheduler's webui provides to create DAG by dragging and dropping, which is not friendly in the face of business scenarios with complex correlations. 3.There will be some unique problems for the offline data warehouse vertical. We developed a more suitable development platform for offline data warehouse based on dolphinscheduler, a scheduling engine. 4.Based on the above, we have developed a new platform to achieve: complex Hql automated generation scheduling, data dependency support, more powerful periodic replenishment, more convenient alerts, clearer log display, and realized the support of data pedigree and data insight. Currently, we are also developing the ability to generate and schedule FlinkSQL pedigree analysis with one click
5.The newly developed platform is designed at the beginning to be compatible with multiple versions, so it is completely decoupled from dolphinscheduler, only the packaged jar, put into the lib directory of dolphinscheduler, and it is done ready. 6.I would like to contribute it as a sub-project of dolphinscheduler, positioning: the offline data warehouse vertical, to achieve a common offline data warehouse solution in the industry, to meet the needs of users to build offline data warehouses. Related links:https://github.com/apache/dolphinscheduler/issues/12971
