kosteev commented on PR #42410:
URL: https://github.com/apache/airflow/pull/42410#issuecomment-2462272348
Not pretending that this is very practical case, however in Cloud Composer
we saw this (and not only once):
Imagine that you give customer an example DAG that looks like this:
<pre>
...
dag = DAG(
'dag_id',
default_args=default_args,
schedule_interval='*/10 * * * *',
)
# task with highest priority
t1 = BashOperator(
task_id='task_id',
bash_command='echo test',
dag=dag,
priority_weight=2**31 - 1)
</pre>
Customer modifies DAG and adds extra task `t2` by copying `t1` and setting
dependencies between them:
<pre>
...
dag = DAG(
'dag_id',
default_args=default_args,
schedule_interval='*/10 * * * *',
)
# task with highest priority
t1 = BashOperator(
task_id='task_id',
bash_command='echo test',
dag=dag,
priority_weight=2**31 - 1)
t2 = BashOperator(
task_id='task_id2',
bash_command='echo test2',
dag=dag,
priority_weight=2**31 - 1)
t1 >> t2
</pre>
Then this DAG will cause an issue and break scheduler (because t2
priority_weight will overflow).
Btw, I found and example DAG like this on stackoverflow
https://stackoverflow.com/questions/66098050/airflow-dag-not-triggered-at-schedule-time.
I am not saying that this is at all common, but it is very unexpected for
user to have scheduler broken after slight modification of the DAG like this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
