potiuk commented on PR #58578: URL: https://github.com/apache/airflow/pull/58578#issuecomment-3567011708
There are lots of parallells - I think there is one difference: 1) I see a big value in actually doing some manual verification between the steps. There are some things we find out and verify when there are files produced, when they are built - you can always manually intervene between the steps and modify things for each step 2) When you do everything in one-go, there is this big potential issue "did I check anything at all" ? I.e if we automate check, who is checking if the automation still works ? Are we sure that the green status is not maybe because the script just returned "OK" (regardless of actual checks) ? That's why I like much better that manual operations are stil driving the checks - and that you have a clear description of what happens after each step - those files are created, those files are verified, we check out the repo etc. This gives enough "check-points" that it's likely someone who is doing those check might have a second thought and pause and do some "ls" or `git diff` at places when they have doubts. I think our goal should not be "full autoamation that might give false sense of correctness" but to give a series of steps that are easy to follow and that have easy to verify manual intermediate outputs, that could be good and frequent checkpoints. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
