Usiel opened a new pull request, #32511: URL: https://github.com/apache/superset/pull/32511
### SUMMARY The `prune_query` task fails because `prune_query.request.properties` has no entry for `retention_period_days` when using a RabbitMQ broker. The task works fine when using sqlite as a broker. From what I could gather from celery's docs (e.g. https://docs.celeryq.dev/en/latest/userguide/periodic-tasks.html#available-fields) `args` or `kwargs` should be the preferred mechanism to pass args to a task. `options` is to be used to override `apply_async` args (e.g. `queue`, `priority`, ...). The fact that a sqlite broker passes any option to the task seems more like an oversight. The change makes sure that people who have been happily using sqlite with the old beat schedule are not impacted (besides a new warning log). ``` superset-worker-dev-0 worker [2025-03-05 06:49:10,476: ERROR/MainProcess] Task prune_query[6f48cb1c-48fb-498a-8fca-f28f95ea296d] raised unexpected: TypeError('unsupported type for timedelta days component: NoneType') superset-worker-dev-0 worker Traceback (most recent call last): superset-worker-dev-0 worker File "/opt/.pyenv/versions/3.11.4/envs/superset/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task superset-worker-dev-0 worker R = retval = fun(*args, **kwargs) superset-worker-dev-0 worker ^^^^^^^^^^^^^^^^^^^^ superset-worker-dev-0 worker File "/opt/.pyenv/versions/3.11.4/envs/superset/lib/python3.11/site-packages/superset/initialization/__init__.py", line 113, in __call__ superset-worker-dev-0 worker return task_base.__call__(self, *args, **kwargs) superset-worker-dev-0 worker ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ superset-worker-dev-0 worker File "/opt/.pyenv/versions/3.11.4/envs/superset/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__ superset-worker-dev-0 worker return self.run(*args, **kwargs) superset-worker-dev-0 worker ^^^^^^^^^^^^^^^^^^^^^^^^^ superset-worker-dev-0 worker File "/opt/.pyenv/versions/3.11.4/envs/superset/lib/python3.11/site-packages/superset/tasks/scheduler.py", line 134, in prune_query superset-worker-dev-0 worker ).run() superset-worker-dev-0 worker ^^^^^ superset-worker-dev-0 worker File "/opt/.pyenv/versions/3.11.4/envs/superset/lib/python3.11/site-packages/superset/commands/sql_lab/query.py", line 63, in run superset-worker-dev-0 worker < datetime.now() - timedelta(days=self.retention_period_days) superset-worker-dev-0 worker ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ superset-worker-dev-0 worker TypeError: unsupported type for timedelta days component: NoneType ``` ### TESTING INSTRUCTIONS #### RabbitMQ Test Pre: Set up RabbitMQ (https://docs.celeryq.dev/en/stable/getting-started/backends-and-brokers/rabbitmq.html#) 1. Configure Superset to use the RabbitMQ broker (`broker_url = "amqp://...`) 2. Uncomment the `prune_query` schedule 3. Set the schedule to run every minute Expected: `prune_query` succeeds #### Backwards Compatibility Test 1. Uncomment the `prune_query` schedule 2. Change `kwargs` to `options` to replicate the old schedule 3. Set the schedule to run every minute Expected: `prune_query` succeeds but logs a warning ### ADDITIONAL INFORMATION <!--- Check any relevant boxes with "x" --> <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue --> - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API None of the above -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
