[
https://issues.apache.org/jira/browse/AIRFLOW-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessio Palma updated AIRFLOW-2610:
-----------------------------------
Description:
I'm running the PythonBranchOperator inside a scheduled DAG, sadly things do
not seem to work. I'm getting incomplete executions, here there are some
examples:
!Screen Screen Shot 2018-06-13 at 11.05.13 AM.png!
Usually, when a dag_run completes, the scheduler writes a row where states the
status which has been set for it; something like:
INFO - Marking run <DagRun BRANCH_OPERATOR_TEST5 @ 2018-06-13 12:45:00:
scheduled__2018-06-13T12:45:00, externally triggered: False> successful
for the above Dags, this does not happen.
I noticed that when the Dag is manually started or triggered by another Dag, it
works perfectly. In this case, the PythonBranchOperator has no issues.
I added a tree view of the executions.
On the left, there are the executions which have been triggered by another Dag,
in the center, there are the scheduled executions, on the right, there are the
executions triggered manually.
You can note some scheduled execution which completed with success, actually,
these have been cleared after they failed.
!Screen Shot 2018-06-13 at 2.38.37 PM.png!
I think there is an issue on the scheduler; but currently, it is hard for me to
debug the issue.
For the sake of the completeness, show chains work also in scheduled mode, for
example:
!Screen Shot 2018-06-13 at 11.05.13 AM.png!
works perfectly:
!Screen Shot 2018-06-13 at 3.22.43 PM.png!
Any idea, any quick fix to this issue?
was:
I'm running the PythonBranchOperator inside a scheduled DAG, sadly things do
not seem to work. I'm getting incomplete executions, here there are some
examples:
!Screen Shot 2018-06-13 at 2.38.37 PM.png!
Usually, when a dag_run completes, the scheduler writes a row where states the
status which has been set for it; something like:
INFO - Marking run <DagRun BRANCH_OPERATOR_TEST5 @ 2018-06-13 12:45:00:
scheduled__2018-06-13T12:45:00, externally triggered: False> successful
for the above Dags, this does not happen.
I noticed that when the Dag is manually started or triggered by another Dag, it
works perfectly. In this case, the PythonBranchOperator has no issues.
I added a tree view of the executions.
On the left, there are the executions which have been triggered by another Dag,
in the center, there are the scheduled executions, on the right, there are the
executions triggered manually.
You can note some scheduled execution which completed with success, actually,
these have been cleared after they failed.
!Screen Shot 2018-06-13 at 2.38.37 PM.png!
I think there is an issue on the scheduler; but currently, it is hard for me to
debug the issue.
For the sake of the completeness, show chains work also in scheduled mode, for
example:
!Screen Shot 2018-06-13 at 11.05.13 AM.png!
works perfectly:
!Screen Shot 2018-06-13 at 3.22.43 PM.png!
Any idea, any quick fix to this issue?
> AIRFLOW PythonBranchOperator
> ----------------------------
>
> Key: AIRFLOW-2610
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2610
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 1.9.0
> Reporter: Alessio Palma
> Priority: Major
> Attachments: Screen Shot 2018-06-13 at 11.05.13 AM.png, Screen Shot
> 2018-06-13 at 11.08.26 AM.png, Screen Shot 2018-06-13 at 2.38.37 PM.png,
> Screen Shot 2018-06-13 at 3.22.43 PM.png
>
>
> I'm running the PythonBranchOperator inside a scheduled DAG, sadly things do
> not seem to work. I'm getting incomplete executions, here there are some
> examples:
> !Screen Screen Shot 2018-06-13 at 11.05.13 AM.png!
> Usually, when a dag_run completes, the scheduler writes a row where states
> the status which has been set for it; something like:
> INFO - Marking run <DagRun BRANCH_OPERATOR_TEST5 @ 2018-06-13 12:45:00:
> scheduled__2018-06-13T12:45:00, externally triggered: False> successful
> for the above Dags, this does not happen.
> I noticed that when the Dag is manually started or triggered by another Dag,
> it works perfectly. In this case, the PythonBranchOperator has no issues.
> I added a tree view of the executions.
> On the left, there are the executions which have been triggered by another
> Dag, in the center, there are the scheduled executions, on the right, there
> are the executions triggered manually.
> You can note some scheduled execution which completed with success, actually,
> these have been cleared after they failed.
> !Screen Shot 2018-06-13 at 2.38.37 PM.png!
> I think there is an issue on the scheduler; but currently, it is hard for me
> to debug the issue.
>
> For the sake of the completeness, show chains work also in scheduled mode,
> for example:
> !Screen Shot 2018-06-13 at 11.05.13 AM.png!
> works perfectly:
> !Screen Shot 2018-06-13 at 3.22.43 PM.png!
> Any idea, any quick fix to this issue?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
