TJaniF opened a new issue, #48629:
URL: https://github.com/apache/airflow/issues/48629
### Apache Airflow version
main (development)
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
Was testing a dag that makes API calls in a dynamically mapped task to an
API that does not allow any parallel queries and got rate limited repeatedly.
Tested it in a toy dag and it looks like the `max_active_tis_per_dag=1` is not
respected right now, causing several mapped task instances to run in parallel.
Dag to test:
```
from airflow.sdk import dag
from airflow.decorators import task
from datetime import datetime
@dag()
def max_ti_test():
@task
def upstream():
return [1,2,3,4,5,6,7,8,9,10]
@task(max_active_tis_per_dag=1)
def mapped_task(x):
import time
time.sleep(10)
print(datetime.now()) # these in the logs should be at least 10
seconds apart!
return x
mapped_task.expand(x=upstream())
max_ti_test()
```
### What you think should happen instead?
Only one of the dynamically mapped task instances should run at the same
time.
### How to reproduce
Run the dag above, see several of the dtm tasks running at the same time,
then check the logs to find near identical timestamps.
### Operating System
MacOs
### Versions of Apache Airflow Providers
None, well, on main/breeze so actually all of them. But none relevant.
### Deployment
Other
### Deployment details
Breeze
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
