Some issues remain:
* one_failed not executed as dag run is marked failed seemingly prematurely
(@chris yes you should see this, see below for an example that is not working
properly), confirmed regression
* celery instability Alex
* Wrong DAG state after failure inside branch
So Alpha 2 is definitely not ready for production, but please do put in your
canary dags and let them run. I am still quite concerned about the scheduler
integrity and stability.
- Bolke
one_failed_not_executed.py
====
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.dummy_operator import DummyOperator
from datetime import datetime, timedelta
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'start_date': datetime(2016,10,5,19),
'email': ['[email protected]'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(seconds=1),
}
dag = DAG('tutorial', default_args=default_args, schedule_interval='@once')
task1 = BashOperator(
task_id='first_one',
bash_command='date',
dag=dag)
task2 = BashOperator(
task_id='second_one',
bash_command='this_should_not_work',
dag=dag)
task2.set_upstream(task1)
task3 = BashOperator(
task_id='third_one',
bash_command='random_command_third',
dag=dag)
task3.set_upstream(task2)
fail_task = DummyOperator(
task_id='one_failed',
trigger_rule='one_failed',
dag=dag)
fail_task.set_upstream([task1,task2,task3])
> On 4 Jan 2017, at 20:35, Chris Riccomini <[email protected]> wrote:
>
> Bolke, can you describe the current state of the alpha 2 release? I saw
> some comments from Alex yesterday about celery instability. If I'm running
> on LocalExecutor, should I be seeing any issues?
>
> On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <[email protected]> wrote:
>
>> Hi All,
>>
>> I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~bolke/ <
>> https://people.apache.org/~bolke/> .
>>
>> Note: This still cannot be considered an Apache release. Working on this.
>>
>> This build is signed (note it is served over https).
>>
>> Changes are in the area of scheduler stability.
>>
>> - Bolke
