rmasters opened a new pull request, #28706: URL: https://github.com/apache/superset/pull/28706
<!--- Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/ Example: fix(dashboard): load charts correctly --> ### SUMMARY Fixes #28705 We found our chart cache warm-up task (as in [the example in the Kubernetes docs](https://superset.apache.org/docs/installation/kubernetes#configure-the-appropriate-celery-jobs-and-smtpslack-settings)) was failing with this error: ``` [2024-05-24 22:30:00,045: INFO/ForkPoolWorker-1] cache-warmup[ae85f28c-acfa-4bb1-a885-e7f0121610b9]: Loading strategy [2024-05-24 22:30:00,045: INFO/ForkPoolWorker-1] cache-warmup[ae85f28c-acfa-4bb1-a885-e7f0121610b9]: Loading TopNDashboardsStrategy [2024-05-24 22:30:00,046: INFO/ForkPoolWorker-1] cache-warmup[ae85f28c-acfa-4bb1-a885-e7f0121610b9]: Success! HTTPException Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1788, in dispatch_request self.raise_routing_exception(req) File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1770, in raise_routing_exception raise request.routing_exception # type: ignore File "/usr/local/lib/python3.10/site-packages/flask/ctx.py", line 351, in match_request result = self.url_adapter.match(return_rule=True) # type: ignore File "/usr/local/lib/python3.10/site-packages/werkzeug/routing/map.py", line 619, in match raise MethodNotAllowed(valid_methods=list(e.have_match_for)) from None werkzeug.exceptions.MethodNotAllowed: 405 Method Not Allowed: The method is not allowed for the requested URL. [repeated] ``` On investigation I found that: - The `cache-warmup` task calls `PUT /superset/warm_up_caches/` (via the `fetch_url` task) - `/superset/warm_up_caches/` (`Superset.warm_up_caches`) exists but is a GET request - That endpoint is deprecated by `/api/v1/chart/warm_up_caches` (`ChartRestApi.warm_up_cache`) (scheduled for 4.0 but I couldn't find discussions around that) This PR changes the task to call the newer endpoint (accidentally reverted in 56069b05f9cf4d0c725d1b4b0ad6038b50837cd4). I also ran into CSRF issues (similar to discussion in #27160), so added a method to fetch this (thanks to Will Gan on slack for the hint). Perhaps this can be placed more centrally? This appears to work on installations with CSRF disabled (I've tested with `WTF_CSRF_ENABLED = False`). <!--- Describe the change below, including rationale and design decisions --> ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF <!--- Skip this if not applicable --> ### TESTING INSTRUCTIONS <!--- Required! What steps can be taken to manually verify the changes? --> 1. Ensure you have a scheduled job to warmup caches, e.g. by adding to CeleryConfig: ```diff class CeleryConfig: broker_url = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}" - imports = ("superset.sql_lab",) + imports = ( + "superset.sql_lab", + "superset.tasks.cache", + ) result_backend = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}" worker_prefetch_multiplier = 1 task_acks_late = False @@ -87,6 +90,13 @@ class CeleryConfig: "task": "reports.prune_log", "schedule": crontab(minute=10, hour=0), }, + "cache-warmup-dummy": { + "task": "cache-warmup", + "schedule": crontab(minute="*/1", hour="*"), + "kwargs": { + "strategy_name": "dummy", + }, + }, } ``` 2. If starting from an empty installation, create a dashboard, with at least one data-backed chart, and publish it 3. Each minute you should see log lines like: ``` superset_worker | [2024-05-25 00:18:03,203: INFO/ForkPoolWorker-1] fetch_url[b9a9c5c6-41e0-491f-a8cb-d49c0b9ab396]: Fetching http://superset_app:8088/api/v1/security/csrf_token/ superset_app | 2024-05-25 00:18:03,198:DEBUG:superset.stats_logger:[stats_logger] (timing) ChartRestApi.warm_up_cache.time | 176.14193800545763 superset_app | 2024-05-25 00:18:03,200:INFO:werkzeug:192.168.165.7 - - [25/May/2024 00:18:03] "PUT /api/v1/chart/warm_up_cache HTTP/1.1" 200 - ``` as compared to this on master: ``` superset_worker | [2024-05-25 00:26:00,162: INFO/MainProcess] Task fetch_url[a488cfb6-1d63-42bb-be00-a3e6af3e2c4a] received superset_worker | [2024-05-25 00:26:00,165: INFO/ForkPoolWorker-1] fetch_url[9a054a67-acf0-4fd8-b0dd-42ab28da886b]: Fetching http://superset_app:8088/superset/warm_up_cache/ with payload {"chart_id": 156} superset_app | 2024-05-25 00:26:00,168:WARNING:superset.views.base:HTTPException superset_app | Traceback (most recent call last): superset_app | File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request superset_app | rv = self.dispatch_request() superset_app | File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1458, in dispatch_request superset_app | self.raise_routing_exception(req) superset_app | File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1440, in raise_routing_exception superset_worker | [2024-05-25 00:26:00,167: INFO/ForkPoolWorker-7] cache-warmup[57b6894a-e9a2-4fba-bf8e-2e7d74826f61]: Scheduling {"chart_id": 156} superset_app | raise request.routing_exception # type: ignore superset_app | File "/usr/local/lib/python3.10/site-packages/flask/ctx.py", line 353, in match_request superset_app | result = self.url_adapter.match(return_rule=True) # type: ignore superset_app | File "/usr/local/lib/python3.10/site-packages/werkzeug/routing/map.py", line 624, in match superset_app | raise MethodNotAllowed(valid_methods=list(e.have_match_for)) from None superset_app | werkzeug.exceptions.MethodNotAllowed: 405 Method Not Allowed: The method is not allowed for the requested URL. superset_app | 2024-05-25 00:26:00,170:INFO:werkzeug:192.168.165.8 - - [25/May/2024 00:26:00] "PUT /superset/warm_up_cache/ HTTP/1.1" 405 - ``` A clearer example on my clean install (based on 4.0.0): ``` superset_worker_beat | [2024-05-24 23:38:00,065: INFO/MainProcess] Scheduler: Sending due task cache-warmup-hourly (cache-warmup) superset_worker | [2024-05-24 23:38:00,084: INFO/MainProcess] Task cache-warmup[c8b9a20b-3fea-45e5-a9fc-9a2fdcfa7b40] received superset_worker | [2024-05-24 23:38:00,093: INFO/ForkPoolWorker-7] cache-warmup[c8b9a20b-3fea-45e5-a9fc-9a2fdcfa7b40]: Loading strategy superset_worker | [2024-05-24 23:38:00,094: INFO/ForkPoolWorker-7] cache-warmup[c8b9a20b-3fea-45e5-a9fc-9a2fdcfa7b40]: Loading DummyStrategy superset_worker | [2024-05-24 23:38:00,095: INFO/ForkPoolWorker-7] cache-warmup[c8b9a20b-3fea-45e5-a9fc-9a2fdcfa7b40]: Success! superset_worker | [2024-05-24 23:38:00,161: INFO/ForkPoolWorker-7] cache-warmup[c8b9a20b-3fea-45e5-a9fc-9a2fdcfa7b40]: Scheduling {"chart_id": 1} superset_worker | [2024-05-24 23:38:00,181: INFO/MainProcess] Task fetch_url[efd6a561-9c01-455c-aaae-bf8d4e6fb88e] received superset_worker | [2024-05-24 23:38:00,185: INFO/ForkPoolWorker-7] Task cache-warmup[c8b9a20b-3fea-45e5-a9fc-9a2fdcfa7b40] succeeded in 0.09269619999395218s: {'scheduled': ['{"chart_id": 1}'], 'errors': []} superset_worker | [2024-05-24 23:38:00,202: INFO/ForkPoolWorker-8] fetch_url[efd6a561-9c01-455c-aaae-bf8d4e6fb88e]: Fetching http://webserver.superset.local:8088/api/v1/security/csrf_token/ superset_app | 2024-05-24 23:38:00,259:INFO:werkzeug:192.168.214.6 - - [24/May/2024 23:38:00] "GET /api/v1/security/csrf_token/ HTTP/1.1" 200 - superset_worker | [2024-05-24 23:38:00,261: INFO/ForkPoolWorker-8] fetch_url[efd6a561-9c01-455c-aaae-bf8d4e6fb88e]: Fetching http://webserver.superset.local:8088/api/v1/chart/warm_up_cache with payload {"chart_id": 1} superset_app | 2024-05-24 23:38:02,332:INFO:werkzeug:192.168.214.6 - - [24/May/2024 23:38:02] "PUT /api/v1/chart/warm_up_cache HTTP/1.1" 200 - superset_worker | [2024-05-24 23:38:02,333: INFO/ForkPoolWorker-8] fetch_url[efd6a561-9c01-455c-aaae-bf8d4e6fb88e]: Fetched http://webserver.superset.local:8088/api/v1/chart/warm_up_cache with payload {"chart_id": 1}, status code: 200 superset_worker | [2024-05-24 23:38:02,335: INFO/ForkPoolWorker-8] Task fetch_url[efd6a561-9c01-455c-aaae-bf8d4e6fb88e] succeeded in 2.1522701479989337s: {'success': '{"chart_id": 1}', 'response': '{"result":[{"chart_id":1,"viz_error":null,"viz_status":"success"}]} ``` ### ADDITIONAL INFORMATION <!--- Check any relevant boxes with "x" --> <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue --> - [x] Has associated issue: #28705 - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
