EricGao888 commented on issue #10387: URL: https://github.com/apache/dolphinscheduler/issues/10387#issuecomment-1167015046
> IMHO, we could separate this issue into two parts: > > 1. Add source control ability for resource center. > 2. Enable users to review workflow changes. > > For the first one, since resource center are based on HDFS / S3 / ..., we could add a log file in remote storage to store operation log / commit hash code, etc. and combine commit hash code with object name or tag. Use S3 / HDFS read/write interface to interact with this log file to ensure consistency. In that case, we could enable source control not only for txt / sql / sh file, but also for jar / tar and avoiding exploding the remote git repo. > > For the second one, we could add some kind of mapping function to map workflows into python DAGs. Users will get different versions of python DAGs once they create / update their workflows. Based on that, we could add source control with git protocol to enable users to review workflow changes and provide them with better production experience. > > WDYT @complone @xtr1993 @SbloodyS @zhongjiajie @davidzollo To clarify, those generated python DAGs mentioned above are only for review purposes, we do not really need to run those DAGs. Therefore, there's no need to change current code logic and we may just add an assistant feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
