potiuk edited a comment on pull request #12007:
URL: https://github.com/apache/airflow/pull/12007#issuecomment-727249751
FYI. @XD-DENG and @kaxil -> this was not a bug and it has not been fixed.
The thing is that if a job depends on another which is still in progress, the
"queued" job name is not interpolated, because the result of the previous job
is not yet available, and you can put some outputs from the previous job in the
job name.
Not only that, you can put the output of the previous job as input to a
matrix strategy (using fromJson function) and then not only the name, but also
a number of dependent jobs can be changed. We utilise this for example to
generate dynamic matrix of builds - if we see that the PR has
"full-tests-needed" label, we generate matrix with the full set of python
versions, if it does not - we only use default python version in the matrix.
In the above case, simply waiting for PROD image has not finished yet. The
K8S tests are using PROD image to run the tests and until PROD image is being
built - you will see those all "dependent" matrix K8S job as a single one with
"${{ matrix.... }}. Only after the PROD image is ready, this single "matrix"
job will turn into as many jobs as many parameters there are in the matrix and
then all those job names with turn into the "Real" names with those variables
interpolated.
In a way it is similar to Airflow's JINJA templates, that get resolved only
when our tasks are about to start executing. There are many similarities in the
DAG of Giihub Actions and the Airflow ones.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
