iameugenejo commented on issue #15036:
URL: https://github.com/apache/airflow/issues/15036#issuecomment-832294937
this hasn't happened for the past month or so.
The following is the dag with some values redacted.
```from datetime import timedelta
from airflow.models import DAG
from airflow.operators.bash import BashOperator
import sys
import pendulum
from airflow.utils import timezone
DAG_ID = '{REDACTED}'
now = pendulum.now(timezone.utc)
schedule_interval = '0 16 * * *'
start_date = now - timedelta(days=1)
max_active_runs = 1
num_of_tasks = 6
email_ids = '{REDACTED}'
with DAG(
dag_id=DAG_ID,
start_date=start_date,
max_active_runs=max_active_runs,
default_args={
'owner': 'airflow',
'start_date': start_date,
'max_active_runs': max_active_runs,
'email': email_ids,
'email_on_failure': True,
'email_on_retry': True
},
schedule_interval=schedule_interval,
dagrun_timeout=timedelta(seconds=43200), # 6 hours
catchup=False
) as dag:
tasks = []
for i in range(0, num_of_tasks):
tasks.append(BashOperator(
task_id='edms3_'+str(i+1),
retries=10,
retry_delay=timedelta(seconds=60), # 1 minute retry delay
retry_exponential_backoff=True,
max_retry_delay=timedelta(seconds=900), # 15 minutes max retry
delay
do_xcom_push=True, # return the last line from the stdout
bash_command="REDACTED.sh {} {} ".format(int(i),
int(num_of_tasks)),
dag=dag))
if i != 0:
tasks[i-1] >> tasks[i]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
