o-nikolas commented on code in PR #30426:
URL: https://github.com/apache/airflow/pull/30426#discussion_r1207304449
##########
tests/executors/test_celery_executor.py:
##########
@@ -248,6 +249,22 @@ def test_cleanup_stuck_queued_tasks(self, mock_fail):
assert app.control.revoke.called_with("231")
assert mock_fail.called_once()
+ @conf_vars({("celery", "result_backend_engine_options"): '{"pool_recycle":
1800}'})
+ @mock.patch("celery.Celery")
+ def test_result_backend_engine_options(self, mock_celery):
+ import importlib
+
+ from airflow.config_templates import default_celery
+
+ # relaod celery conf to apply the new config
Review Comment:
```suggestion
# reload celery conf to apply the new config
```
##########
airflow/config_templates/config.yml:
##########
@@ -2041,6 +2041,13 @@ celery:
sensitive: true
example: "db+postgresql://postgres:airflow@postgres/airflow"
default: ~
+ result_backend_engine_options:
+ description: |
+ Optional options dictionary to pass to the Celery result backend
SQLAlchemy engine.
Review Comment:
"Optional options" is a mouthful, what about:
```suggestion
Optional configuration dictionary to pass to the Celery result
backend SQLAlchemy engine.
```
##########
tests/executors/test_celery_executor.py:
##########
@@ -248,6 +249,22 @@ def test_cleanup_stuck_queued_tasks(self, mock_fail):
assert app.control.revoke.called_with("231")
assert mock_fail.called_once()
+ @conf_vars({("celery", "result_backend_engine_options"): '{"pool_recycle":
1800}'})
+ @mock.patch("celery.Celery")
+ def test_result_backend_engine_options(self, mock_celery):
+ import importlib
+
+ from airflow.config_templates import default_celery
+
+ # relaod celery conf to apply the new config
+ importlib.reload(default_celery)
+ # reload celery_executor to recreate the celery app with new config
+ importlib.reload(celery_executor)
Review Comment:
This will create a logical conflict (git won't be able to detect) with #31001
Whoever merges first will have to resolve that, but it is a very easy fix
(just reload the utils module instead)
##########
airflow/config_templates/config.yml:
##########
@@ -2041,6 +2041,13 @@ celery:
sensitive: true
example: "db+postgresql://postgres:airflow@postgres/airflow"
default: ~
+ result_backend_engine_options:
Review Comment:
Does this only apply to the DB/SQLAlchemy backend as the description
explains? If so we should change the name to be more specific (and if not we
should change the description wording to be more generic).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
