codeant-ai-for-open-source[bot] commented on code in PR #41133: URL: https://github.com/apache/superset/pull/41133#discussion_r3573320679
########## superset/tasks/export_dashboard_excel.py: ########## @@ -0,0 +1,361 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" +Celery task that exports every chart on a dashboard to a single multi-sheet +``.xlsx`` file, uploads it to S3, and emails the requesting user a pre-signed +download link. + +In ``"data"`` mode the task re-runs each chart's saved query context under the +requesting user, applies the live dashboard filter state, and streams the results +row-by-row into a constant-memory workbook so large dashboards never load all +data at once. In ``"images"`` mode non-table charts are instead rendered to +images (through the same headless path as scheduled reports, reflecting the live +filters) and embedded, while table-like charts stay tabular. +""" + +from __future__ import annotations + +import logging +import os +import tempfile +from datetime import datetime, timedelta, timezone +from typing import Any + +from celery.exceptions import SoftTimeLimitExceeded +from flask import current_app, g + +from superset import db, security_manager +from superset.charts.data.dashboard_filter_context import ( + apply_dashboard_filter_context, + get_dashboard_filter_context, +) +from superset.charts.schemas import ChartDataQueryContextSchema +from superset.commands.chart.data.get_data_command import ChartDataCommand +from superset.commands.distributed_lock.release import ReleaseDistributedLock +from superset.common.chart_data import ChartDataResultFormat, ChartDataResultType +from superset.dashboards.excel_export import email +from superset.dashboards.excel_export.layout import get_charts_in_layout_order +from superset.dashboards.excel_export.screenshot import render_chart_image +from superset.extensions import celery_app +from superset.utils import json, s3 +from superset.utils.core import override_user +from superset.utils.excel_streaming import StreamingXlsxWriter + +logger = logging.getLogger(__name__) + +# Export modes: "data" streams every chart's tabular result (the default, +# unchanged behavior); "images" embeds non-table charts as rendered images and +# keeps only table-like charts tabular. +EXPORT_MODE_DATA = "data" +EXPORT_MODE_IMAGES = "images" + +# Viz types kept as tabular data in image mode; everything else is rendered as an +# image. Operators can override the set via ``EXCEL_EXPORT_TABLE_VIZ_TYPES``. +TABLE_VIZ_TYPES = {"table", "pivot_table_v2", "pivot_table"} + +EXPORT_SOFT_TIME_LIMIT = 600 +EXPORT_HARD_TIME_LIMIT = 660 + +# Namespace + TTL for the per-user+dashboard in-flight lock the API acquires +# before enqueue and this task releases when it settles. The lock uses the +# shared, atomic DistributedLock backend (Redis when configured, the metadata +# DB otherwise) so it actually synchronizes across the web server and workers — +# unlike a plain cache, which is a no-op under the default ``NullCache``. +# The TTL outlives the hard time limit so a worker killed at that limit (which +# skips the ``finally`` release) cannot hold the lock forever; the release in +# ``finally`` is the fast path that frees it as soon as the task settles. +EXPORT_LOCK_NAMESPACE = "excel_export" +EXPORT_LOCK_TTL_SECONDS = EXPORT_HARD_TIME_LIMIT + 60 + + +def export_lock_params(user_id: int, dashboard_id: int) -> dict[str, int]: + """Key parameters identifying the per-user+dashboard in-flight lock.""" + return {"user_id": user_id, "dashboard_id": dashboard_id} + + +class _ChartSkippedError(Exception): + """Signals a chart that could not be exported and should be listed as skipped.""" + + +def _chart_label(chart: Any) -> str: + """Human-readable label for a chart in the skipped-charts list.""" + return f"{chart.id} - {chart.slice_name or ''}".strip() + + +def _record_to_row(record: dict[str, Any], colnames: list[str]) -> list[Any]: + return [record.get(col) for col in colnames] + + +def _table_viz_types() -> set[str]: + """Viz types kept tabular in image mode (config override or built-in default).""" + return current_app.config.get("EXCEL_EXPORT_TABLE_VIZ_TYPES") or TABLE_VIZ_TYPES Review Comment: **Suggestion:** The override logic for table viz types treats an empty configured collection as falsy and silently falls back to the built-in defaults. This makes `EXCEL_EXPORT_TABLE_VIZ_TYPES = set()` (or `[]`) impossible to use, even though `None` is documented as the fallback trigger. Check explicitly for `None` instead of using `or` so an intentional empty override is honored. [incorrect condition logic] <details> <summary><b>Severity Level:</b> Major ⚠️</summary> ```mdx - ⚠️ Export Images to Excel ignores empty EXCEL_EXPORT_TABLE_VIZ_TYPES override. - ⚠️ Table and pivot charts stay tabular despite image-only intent. - ⚠️ Behavior contradicts config comment in superset/config.py:1454-1459. ``` </details> <details> <summary><b>Steps of Reproduction ✅ </b></summary> ```mdx 1. In `superset/config.py:1457`, configure `EXCEL_EXPORT_TABLE_VIZ_TYPES = set()` to override the default table viz types, even though the preceding comment at `superset/config.py:1456` states “Set to None to fall back to the built-in default.” 2. Start Superset and a Celery worker so the dashboard export endpoint `DashboardRestApi.export_xlsx` (`superset/dashboards/api.py:1593-1602`, 1627-1675) can enqueue the Celery task `export_dashboard_excel` (`superset/tasks/export_dashboard_excel.py:260-273`). 3. From the UI or an HTTP client, trigger an images-mode export by calling `POST /api/v1/dashboard/<pk>/export_xlsx/` with a JSON body like `{"mode": "images"}`, which `export_xlsx` parses into `payload["mode"]` and passes through to `export_dashboard_excel` at `superset/dashboards/api.py:1664-73`. 4. When the Celery task runs, `_build_workbook` (in `superset/tasks/export_dashboard_excel.py:181-241`, from the PR diff) iterates dashboard charts and calls `_renders_as_image` (`superset/tasks/export_dashboard_excel.py:49-51`), which uses `_table_viz_types` (`superset/tasks/export_dashboard_excel.py:44-46`); because `_table_viz_types` returns `current_app.config.get("EXCEL_EXPORT_TABLE_VIZ_TYPES") or TABLE_VIZ_TYPES`, the empty set is treated as falsy and replaced by `TABLE_VIZ_TYPES`, so charts with viz types `table`, `pivot_table_v2`, and `pivot_table` remain tabular instead of being rendered as images, ignoring the explicit empty override. ``` </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=c9cd5fe36716470aaa15c17fb122743f&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=c9cd5fe36716470aaa15c17fb122743f&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) *(Use Cmd/Ctrl + Click for best experience)* <details> <summary><b>Prompt for AI Agent 🤖 </b></summary> ```mdx This is a comment left during a code review. **Path:** superset/tasks/export_dashboard_excel.py **Line:** 103:105 **Comment:** *Incorrect Condition Logic: The override logic for table viz types treats an empty configured collection as falsy and silently falls back to the built-in defaults. This makes `EXCEL_EXPORT_TABLE_VIZ_TYPES = set()` (or `[]`) impossible to use, even though `None` is documented as the fallback trigger. Check explicitly for `None` instead of using `or` so an intentional empty override is honored. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41133&comment_hash=07a8c16d86c61075b36840bb147a96eec299475604a9a165343d7a22ba6eb301&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F41133&comment_hash=07a8c16d86c61075b36840bb147a96eec299475604a9a165343d7a22ba6eb301&reaction=dislike'>👎</a> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
