nivdann opened a new issue, #28595: URL: https://github.com/apache/superset/issues/28595
### Bug description I managed to enable my Alerts & Reports as mentioned in documentation, BUT when I try to use the chart report as CSV to send to email, it fails with HTTP 500 as shown in the worker logs. Alerts and Snapshots work and **are sent to my email as expected.** The only thing that doesn't work and fails is CSV charts. ### How to reproduce the bug 1. Go to Alerts & Reports 2. Add new alert (i added a "select 1;" query with trigger condition != 0 to have the trigger, and also choose a example chart with CSV option) 3. Wait for trigger ### Screenshots/recordings   ### Superset version 3.1.3 ### Python version 3.10 ### Node version I don't know ### Browser Not applicable ### Additional context I using helm to deploy to my EKS running AWS. Using Firefox and geckodriver (defaults). ### Superset Config ``` ROW_LIMIT = 5000 ENABLE_PROXY_FIX = True ALERT_REPORTS_NOTIFICATION_DRY_RUN = False FEATURE_FLAGS = { "DYNAMIC_PLUGINS": True, "ALLOW_ADHOC_SUBQUERY": True, "ENABLE_TEMPLATE_PROCESSING": True, "DASHBOARD_VIRTUALIZATION": True, "DRILL_BY": True, "GLOBAL_ASYNC_QUERIES": False, "ALERT_REPORTS": True } SMTP_HOST = os.getenv("SMTP_HOST") SMTP_STARTTLS = ast.literal_eval(os.getenv("SMTP_STARTTLS")) SMTP_SSL = ast.literal_eval(os.getenv("SMTP_SSL")) SMTP_SSL_SERVER_AUTH = ast.literal_eval(os.getenv("SMTP_SSL_SERVER_AUTH")) SMTP_PORT = os.getenv("SMTP_PORT") SMTP_MAIL_FROM = os.getenv("SMTP_MAIL_FROM") SMTP_USER = "" SMTP_PASSWORD = "" from celery.schedules import crontab class CeleryConfig: broker_url = f"redis://{env('REDIS_HOST')}:{env('REDIS_PORT')}/0" imports = ( "superset.sql_lab", "superset.tasks.cache", "superset.tasks.scheduler", ) result_backend = f"redis://{env('REDIS_HOST')}:{env('REDIS_PORT')}/0" worker_prefetch_multiplier = 10 task_acks_late = True task_annotations = { "sql_lab.get_sql_results": { "rate_limit": "100/s", }, } beat_schedule = { "reports.scheduler": { "task": "reports.scheduler", "schedule": crontab(minute="*", hour="*"), }, "reports.prune_log": { "task": "reports.prune_log", "schedule": crontab(minute=0, hour=0), }, } CELERY_CONFIG = CeleryConfig LOG_LEVEL = 'DEBUG' SCREENSHOT_LOCATE_WAIT = 100 SCREENSHOT_LOAD_WAIT = 600 EMAIL_PAGE_RENDER_WAIT = 60 WEBDRIVER_BASEURL = "http://{{ template "superset.fullname" . }}:{{ .Values.service.port }}" WEBDRIVER_BASEURL_USER_FRIENDLY = "http://superset:8088"; SUPERSET_WEBSERVER_TIMEOUT = 300 from selenium.webdriver.common.desired_capabilities import DesiredCapabilities capabilities = DesiredCapabilities.FIREFOX.copy() capabilities['proxy'] = { "proxyType": "DIRECT" # Direct connection, no proxy } WEBDRIVER_CONFIGURATION = {"capabilities": capabilities} WEBDRIVER_OPTION_ARGS = ["--headless"] SECRET_KEY = os.getenv("SUPERSET_SECRET_KEY") ``` ### Worker logs > [2024-05-19 10:58:00,074: INFO/ForkPoolWorker-1] Scheduling alert test eta: 2024-05-19 10:58:00 > [2024-05-19 10:58:00,081: DEBUG/ForkPoolWorker-1] [stats_logger] (incr) reports.execute > [2024-05-19 10:58:00,081: INFO/ForkPoolWorker-1] Executing alert/report, task id: 39937d56-4280-4e16-bde8-011841a62d90, scheduled_dttm: 2024-05-19T10:58:00 > [2024-05-19 10:58:00,082: INFO/ForkPoolWorker-1] session is validated: id 15, executionid: 39937d56-4280-4e16-bde8-011841a62d90 > [2024-05-19 10:58:00,149: INFO/ForkPoolWorker-1] Running report schedule 39937d56-4280-4e16-bde8-011841a62d90 as user admin > [2024-05-19 10:58:00,385: DEBUG/ForkPoolWorker-1] Parsing with sqlparse statement: select 1; > [2024-05-19 10:58:00,394: DEBUG/ForkPoolWorker-1] Database._get_sqla_engine(). Masked URL: awsathena+rest://athena.us-west-2.amazonaws.com/superset_db?s3_staging_dir=s3%3A%2F%2Fdata%2Fquery_results%2F&work_group=primary > [2024-05-19 10:58:00,397: DEBUG/ForkPoolWorker-1] Database._get_sqla_engine(). Masked URL: awsathena+rest://athena.us-west-2.amazonaws.com/superset_db?s3_staging_dir=s3%3A%2F%2Fdata%2Fquery_results%2F&work_group=primary > [2024-05-19 10:58:01,943: INFO/ForkPoolWorker-1] Query for test took 1552.98 ms > [2024-05-19 10:58:01,963: INFO/ForkPoolWorker-1] Getting chart from http://superset:8088/api/v1/chart/2/data/?format=csv&type=post_processed&force=true as user admin > [2024-05-19 10:58:02,074: INFO/ForkPoolWorker-1] header_data in notifications for alerts and reports {'notification_type': 'Alert', 'notification_source': <ReportSourceFormat.CHART: 'chart'>, 'notification_format': 'CSV', 'chart_id': 2, 'dashboard_id': None, 'owners': [Superset Admin]}, taskid, 39937d56-4280-4e16-bde8-011841a62d90 > [2024-05-19 10:58:02,164: DEBUG/ForkPoolWorker-1] Sent an email to ['[email protected]'] > [2024-05-19 10:58:02,352: INFO/ForkPoolWorker-1] Report sent to email, notification content is None > [2024-05-19 10:58:02,352: DEBUG/ForkPoolWorker-1] [stats_logger] (gauge) reports.email.send.ok1 > [2024-05-19 10:58:02,376: ERROR/ForkPoolWorker-1] A downstream exception occurred while generating a report: 39937d56-4280-4e16-bde8-011841a62d90. Failed generating csv HTTP Error 500: Internal Server Error > Traceback (most recent call last): > File "/app/superset/commands/report/execute.py", line 258, in _get_csv_data > csv_data = get_chart_csv_data(chart_url=url, auth_cookies=auth_cookies) > File "/app/superset/utils/csv.py", line 92, in get_chart_csv_data > response = opener.open(chart_url) > File "/usr/local/lib/python3.10/urllib/request.py", line 525, in open > response = meth(req, response) > File "/usr/local/lib/python3.10/urllib/request.py", line 634, in http_response > response = self.parent.error( > File "/usr/local/lib/python3.10/urllib/request.py", line 563, in error > return self._call_chain(*args) > File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain > result = func(*args) > File "/usr/local/lib/python3.10/urllib/request.py", line 643, in http_error_default > raise HTTPError(req.full_url, code, msg, hdrs, fp) > urllib.error.HTTPError: HTTP Error 500: Internal Server Error > > The above exception was the direct cause of the following exception: > > Traceback (most recent call last): > File "/app/superset/tasks/scheduler.py", line 98, in execute > ).run() > File "/app/superset/commands/report/execute.py", line 729, in run > raise ex > File "/app/superset/commands/report/execute.py", line 727, in run > ).run() > File "/app/superset/commands/report/execute.py", line 689, in run > ).next() > File "/app/superset/commands/report/execute.py", line 586, in next > raise first_ex > File "/app/superset/commands/report/execute.py", line 555, in next > self.send() > File "/app/superset/commands/report/execute.py", line 452, in send > notification_content = self._get_notification_content() > File "/app/superset/commands/report/execute.py", line 364, in _get_notification_content > csv_data = self._get_csv_data() > File "/app/superset/commands/report/execute.py", line 262, in _get_csv_data > raise ReportScheduleCsvFailedError( > superset.commands.report.exceptions.ReportScheduleCsvFailedError: Failed generating csv HTTP Error 500: Internal Server Error ### Checklist - [X] I have searched Superset docs and Slack and didn't find a solution to my problem. - [X] I have searched the GitHub issue tracker and didn't find a similar bug report. - [X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
