EricGao888 commented on issue #10387: URL: https://github.com/apache/dolphinscheduler/issues/10387#issuecomment-1166904625
IMHO, we could separate this issue into two parts: 1. Add source control ability for resource center. 2. Enable users to review workflow changes. For the first one, since resource center are based on HDFS / S3 / ..., we could add a log file in remote storage to store operation log / commit hash code, etc. and combine commit hash code with object name or tag. Use S3 / HDFS read/write interface to interact with this log file to ensure consistency. In that case, we could enable source control not only for txt / sql / sh file, but also for jar / tar and avoiding explode the remote git repo. For the second one, we could add some kind of mapping function to map workflow into python dag. Users will get different versions of python dags once they create / update their workflows. Based on that, we could add source control with git protocol to enable users to review workflow changes and provide them with better production experience. WDYT @complone @xtr1993 @SbloodyS @zhongjiajie @davidzollo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
