potiuk commented on pull request #22327: URL: https://github.com/apache/airflow/pull/22327#issuecomment-1074248212
> Thanks, @potiuk. I am retrieving all the files that are modified in the PR. Let me know now if the statement bellow is in the right way. > > Process: Steps 1, 2, and 3 will be in the Selective Check main script. The other steps will be independent scripts. This is just an example of many other python calls. > > Also, I created another step in ci.yaml for selective_ci_checks.py > > https://github.com/apache/airflow/runs/5619304320?check_suite_focus=true#step:6:587 Good start. I think there are lots of good things there :). One thing though I thin we should improve 1) I think all the checks should still be in on file "selective_checks.py". This will make it easier to manage and share code. 2) This `selective_checks.py` should be run through entrypoint (`airflow-selective-checks`) - very similarly to `airlflow-freespace` 3) Then in CI instead of running `./scripts/ci/....` you will run `airlflow-selective-checks` same as freespace :) 4) Now. I think the clue here is to get diffrent commands in this one `airflow-selective-checks`). I can imagine such commands: * `airflow-selective-checks matrix-strategy` This one (based on the checks) should either print : * python-versions: * `::set-output name=python-versions::[''3.7', '3.8', '3.9', '3.10']` - if we determine that all python versions should be used * `::set-output name=python-versions::[''3.7']` - if we determine that only this one python version should be used Plus few more outputs that are related: * python-versions-list-as-string * kubernetes-versions * kubernetes-versions-list-as-string * all-python-versions * mysql-versions * mssql-versions * default-mssql-version * ..... * postgres-exclude * mysql-exclude And so on - basically all the parameters that determine what should be the matrix of tests executed. Then the next step will be to build sther commands: * `airflow-selective-checks image-build-needed` `::set-output name=image-build::true` or `::set-output name=image-build::true` depending if we determine that images are needed or not * `airflow-selective-checks doc-build-needed` similarly but for `docs-build` * `airflow-selective-checks test-types` Similarly but for `test-types` But we can do it gradually. The `matrix-strategy` seems to be good start, I hope it makes sense :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
