[jira] [Commented] (AIRFLOW-4428) Externally triggered DagRun marked as 'success' state but all tasks for it are 'None' state
[
https://issues.apache.org/jira/browse/AIRFLOW-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16861883#comment-16861883
]
Ramkumar commented on AIRFLOW-4428:
---
I installed Airflow 1.10.3 using Docker and facing the same issue as described
by Leonardo. Task's are executed for the first automated run of the DAG. When i
trigger the DagRun manually it is marked as 'success' however the tasks are
skipped or in None state. I believe i am missing something since the basic
scheduling feature doesn't works in 1.10.3. Is there any workaround ? In 1.10.2
the same DAG works.
> Externally triggered DagRun marked as 'success' state but all tasks for it
> are 'None' state
> ---
>
> Key: AIRFLOW-4428
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4428
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG, DagRun, scheduler
>Affects Versions: 1.10.3
> Environment: LocalExecutor, 16cpu amazon linux, airflow 1.10.3, mysql
> RDS meta
>Reporter: t oo
>Priority: Critical
> Fix For: 1.10.4
>
> Attachments: Screenshot from 2019-05-10 10-42-53.png, graph.png
>
>
> Dagrun status showing success for a given execdate but:
> * all tasks are white in graph/tree view
> * on cli running airflow task_state for any of the tasks showing in the
> graph/tree view returns None
> * taskinstance view shows no tasks for the given dagid
> This dag has 800 tasks, non_pooled_task_slot_count is 2000, max_tis_per_query
> is 2000, no pools created/assigned to the dag
> This is an externally triggered dag, all my dags are purely externally
> triggered.
> For execdates 20150101, 20160101, 20170101 getting this issue but 20180101
> works. Maybe it has something to do with this line in my dag mentioning 2017
> ---> default_args = {'owner': 'airflow','start_date': dt.datetime(2017, 6,
> 1),'retries': 0,'retry_delay': dt.timedelta(minutes=5),}
> As per comments below this issue was introduced in 1.10.3. release, i suspect
> the below jira caused it:
> * https://github.com/apache/airflow/pull/4808/files
> (https://issues.apache.org/jira/browse/AIRFLOW-3982)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-4814) Dag marked success but tasks are with NO status
Ramkumar created AIRFLOW-4814:
-
Summary: Dag marked success but tasks are with NO status
Key: AIRFLOW-4814
URL: https://issues.apache.org/jira/browse/AIRFLOW-4814
Project: Apache Airflow
Issue Type: Bug
Components: scheduler
Affects Versions: 1.10.3
Reporter: Ramkumar
Attachments: DAG_Fargate.py, image-2019-06-19-18-03-24-722.png,
image-2019-06-19-18-20-53-814.png
I installed Airflow 1.10.3 with Celery executor. I am experiencing abnormal
issues with the scheduler. This issue is also reported by some other users in
AIRFLOW-4428 and the resolution comments doesn't work for me.
# Define the DAG
dag = DAG(
dag_id='DAG_TMap_1_ECS_FG',
default_args={"owner": "airflow", *"start_date": datetime.utcnow(),}*,
schedule_interval='0 */5 * ? * *',
)
*Steps*
a) I created a DAG with ECS Operator and 2 different schedule expressions with
2 different results.
*Test 1 : Schedule_interval = '0/5 * * * *'*
The issue i am experiencing
a) DAGs are not automatically triggered by the scheduler.
b) If I ran the DAG manually, the tasks in it are with NO status , DAG status
is success.
!image-2019-06-19-18-03-24-722.png!
*Test 2: Schedule_interval = '@once'*
The behavior i am experiencing
a) DAGs are automatically triggered once by the scheduler. All the tasks
executed
b) If I ran the DAG manually, the tasks in it are with NO status , DAG status
is success.
!image-2019-06-19-18-20-53-814.png!
Could you give me a hint what i am doing wrong. I am using 1.10.3 for testing
ECS fargate.
In 1.10.2 i dont have these issues.
The issue also happens with other operators.
Also attached my DAG defintion
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-5228) Questions on Airflow licensing and support
Ramkumar created AIRFLOW-5228: - Summary: Questions on Airflow licensing and support Key: AIRFLOW-5228 URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 Project: Apache Airflow Issue Type: Task Components: scheduler Affects Versions: 1.10.3 Reporter: Ramkumar Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i can create Jiras. However my customer want to get the support statements. Can you provide the support statement? a) SLAs, response times for the jiras ? b) Any other Support Level Agreements? c) when a version considered end-of-life & retired? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (AIRFLOW-5228) support statements for Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar updated AIRFLOW-5228: -- Summary: support statements for Airflow (was: Questions on Airflow licensing and support) > support statements for Airflow > --- > > Key: AIRFLOW-5228 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 > Project: Apache Airflow > Issue Type: Task > Components: scheduler >Affects Versions: 1.10.3 >Reporter: Ramkumar >Priority: Major > > Hallo I am working on an Airflow POC. I understand Airflow is an Open source > project and i can create Jiras. > However my customer want to get the support statements. Can you provide the > support statement? > a) SLAs, response times for the jiras ? > b) Any other Support Level Agreements? > c) when a version considered end-of-life & retired? > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (AIRFLOW-5228) support statements for Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar updated AIRFLOW-5228: -- Description: Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i can create Jiras. However my customer want to get the support statements. Can you provide the support statement? a) SLAs, response times for the jiras ? b) Any other Support Level Agreements? c) when a version considered end-of-life & retired? d) is support provided with Cloud Managed services like google composer was: Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i can create Jiras. However my customer want to get the support statements. Can you provide the support statement? a) SLAs, response times for the jiras ? b) Any other Support Level Agreements? c) when a version considered end-of-life & retired? > support statements for Airflow > --- > > Key: AIRFLOW-5228 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 > Project: Apache Airflow > Issue Type: Task > Components: scheduler >Affects Versions: 1.10.3 >Reporter: Ramkumar >Priority: Major > > Hallo I am working on an Airflow POC. I understand Airflow is an Open source > project and i can create Jiras. > However my customer want to get the support statements. Can you provide the > support statement? > a) SLAs, response times for the jiras ? > b) Any other Support Level Agreements? > c) when a version considered end-of-life & retired? > d) is support provided with Cloud Managed services like google composer > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (AIRFLOW-5228) support statements for Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar updated AIRFLOW-5228: -- Description: Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i am able to create Jiras. However my customer want to get the support statements for Airflow. a) Can you guide me for getting the support statement? b) SLAs, response times for the jiras ? c) other Support Level Agreements? d) when a version considered end-of-life & retired? e) Is support provided with Cloud services like google composer ? f) For bug fixes patches are provided ? How can we patch the bug fixes? g) Can you provide any community forums or groups which are open for discussions on Airflow? was: Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i can create Jiras. However my customer want to get the support statements. Can you provide the support statement? a) SLAs, response times for the jiras ? b) Any other Support Level Agreements? c) when a version considered end-of-life & retired? d) is support provided with Cloud Managed services like google composer > support statements for Airflow > --- > > Key: AIRFLOW-5228 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 > Project: Apache Airflow > Issue Type: Task > Components: scheduler >Affects Versions: 1.10.3 >Reporter: Ramkumar >Priority: Major > > Hallo I am working on an Airflow POC. I understand Airflow is an Open source > project and i am able to create Jiras. > However my customer want to get the support statements for Airflow. > a) Can you guide me for getting the support statement? > b) SLAs, response times for the jiras ? > c) other Support Level Agreements? > d) when a version considered end-of-life & retired? > e) Is support provided with Cloud services like google composer ? > f) For bug fixes patches are provided ? How can we patch the bug fixes? > g) Can you provide any community forums or groups which are open for > discussions on Airflow? > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
