Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
raphaelauv commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2322832278 no stale -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
github-actions[bot] commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-232279 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
raphaelauv commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2186178224 Already did -> https://github.com/apache/airflow/issues/37332#issuecomment-2018836100 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
ephraimbuddy commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2186116551 > when a task is skipped, airflow mark the task it FAILED This is strange and I doubt a skipped task is marked Failed. Please without a clear illustration, this issue is confusing. Can you make a short screen capture video on this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
raphaelauv commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2185964000 when a task is skipped, airflow mark the task it FAILED when a DTM task have a subtask skipped, airflow mark the task SUCCESS --- `What if you click to see the logging` I can't know from the UI which DTM task have a subtask that was skipped -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
ephraimbuddy commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2185949379 What if you click to see the logging? Do you see failed tasks/skipped that way? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
raphaelauv commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2185900891  If I have 10 DTM tasks like toto ( tata , tutu ... ) I can't know where a task was skipped cause the DTM status is success -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
ephraimbuddy commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2185876092 > the issue is that the task status is not in fail ( which is the case when not using DTM ), so if I have 10 DTM tasks I can't know where a task was skipped From the above, all the running tasks would be skipped(SKIPPED state). Do you mean that in dynamic mapped task, the states for the unfinished this are not in SKIPPED? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
raphaelauv commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2185868098 yes the dag_run is in fail it's good , the issue is not there the issue is that the task status is not in fail ( which is the case when not using DTM ), so if I have 10 DTM tasks I can't know where a task was skipped -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
ephraimbuddy commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2185832766 @raphaelauv https://github.com/apache/airflow/blob/9901a065fcd93307d8e1d69e34621966d7313511/airflow/jobs/scheduler_job_runner.py#L1444-L1455. The dagrun timeout you have on your dag causes the dag to fail. You can read it on the link above -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
raphaelauv commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2185822456 hey @ephraimbuddy , could you please update the tags of the issue , thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
raphaelauv commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2044184401 @ephraimbuddy could you please remo6the tag wait for response , thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
github-actions[bot] commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2043917725 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
raphaelauv commented on issue #37332:
URL: https://github.com/apache/airflow/issues/37332#issuecomment-2018836100
I don''t understand , what is your question ?
--
also the problem is not present without dynamic task mapping
```python
from datetime import timedelta
from airflow.operators.python import PythonOperator
from pendulum import today
from airflow import DAG
dag = DAG(
dag_id="test_dag",
schedule_interval=None,
dagrun_timeout=timedelta(seconds=5),
start_date=today("UTC").add(days=-1)
)
with dag:
def something(arg):
import time
time.sleep(int(arg))
PythonOperator(
task_id="toto",
python_callable=something,
op_args="10")
PythonOperator(
task_id="tata",
python_callable=something,
op_args="10")
``
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
ephraimbuddy commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2018498579 It's actually there that it has timed out -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
ephraimbuddy commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2018474312 The problem here is that you have a 5-second timeout on the DAG. Not sure why it didn't log but I'll investigate. By definition, skipped is a success state see https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html#dag-run-status and dag should be marked success but because it timeout, it marked it as failed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]
raphaelauv opened a new issue, #37332:
URL: https://github.com/apache/airflow/issues/37332
### Apache Airflow version
2.8.1
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
this is confusing and not obvious to find the problem when just looking at
the grid view on a complex dag
### What you think should happen instead?
maybe we should mark the task as skipped or failed
### How to reproduce
```python
from datetime import timedelta
from airflow.operators.python import PythonOperator
from pendulum import today
from airflow import DAG
dag = DAG(
dag_id="test_dag",
schedule_interval=None,
dagrun_timeout=timedelta(seconds=5),
start_date=today("UTC").add(days=-1)
)
with dag:
def something(arg):
import time
time.sleep(int(arg))
PythonOperator.partial(
task_id="toto",
python_callable=something,
).expand(op_args=[("10",), ("5",), ("4",)])
```
### Operating System
ubuntu 22.04
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
