[GitHub] [airflow] kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable
kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make
send_task_to_executor timeout configurable
URL: https://github.com/apache/airflow/pull/7143#discussion_r370939852
##
File path: airflow/executors/celery_executor.py
##
@@ -41,6 +41,8 @@
CELERY_SEND_ERR_MSG_HEADER = 'Error sending Celery task'
+OPERATION_TIMEOUT = conf.getint('celery', 'operation_timeout')
Review comment:
Can we add a test for this? Let's say a user change this setting to
`conf.get('celery', 'operation_timeout')` that test should fail.
Also let's add a fallback value.
```
OPERATION_TIMEOUT = conf.getint('celery', 'operation_timeout', fallback=2)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable
kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable URL: https://github.com/apache/airflow/pull/7143#discussion_r370939414 ## File path: airflow/config_templates/config.yml ## @@ -1159,6 +1159,14 @@ type: string example: ~ default: "prefork" +- name: operation_timeout + description: | +The number of seconds to wait before timing out send task or fetch task state operations Review comment: ```suggestion The number of seconds to wait before timing out ``send_task`` (``fetch_celery_task_state``) or ``fetch_task`` (``send_task_to_executor``) state operations ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable
kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable URL: https://github.com/apache/airflow/pull/7143#discussion_r369470733 ## File path: airflow/config_templates/default_airflow.cfg ## @@ -537,6 +537,9 @@ ssl_cacert = # https://docs.celeryproject.org/en/latest/userguide/concurrency/eventlet.html pool = prefork +# The number of seconds to wait before timing out apply_async +connection_timeout = 2 Review comment: Is it actually a `connection_timeout`? Trying to think if we can call it something specific to apply_async This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable
kaxil commented on a change in pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable URL: https://github.com/apache/airflow/pull/7143#discussion_r369470809 ## File path: airflow/config_templates/default_airflow.cfg ## @@ -537,6 +537,9 @@ ssl_cacert = # https://docs.celeryproject.org/en/latest/userguide/concurrency/eventlet.html pool = prefork +# The number of seconds to wait before timing out apply_async +connection_timeout = 2 Review comment: @ashb what do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
