parag2131 opened a new issue, #37127:
URL: https://github.com/apache/superset/issues/37127
### Bug description
#Summary
When running Apache Superset 6.0.0 using Docker and attempting to enable
alerts / scheduled email reports, starting a separate Celery Beat container
causes Superset to fail with permission errors related to celerybeat-schedule.
This is confusing because previous Superset versions required a Beat
service, while Superset 6.0.0 no longer needs Celery Beat, but this is not
clearly documented and leads to repeated runtime failures.
Deployment type
Docker Compose
Redis as broker
PostgreSQL as metadata DB
Celery enabled for alerts/reports
SMTP configured for email reports
What I was trying to do
Enable Scheduled Alerts & Email Reports using Celery with Redis, following
older Superset deployment patterns by running:
Superset webserver
Celery worker
Celery beat (scheduler)
#docker-compose.yml (Superset 6.0.0) :
services:
db_test:
image: postgres:15
container_name: superset_postgres_test
environment:
POSTGRES_DB: superset_test
POSTGRES_USER: superset_test
POSTGRES_PASSWORD: superset_test_pwd
volumes:
- superset_test_pgdata:/var/lib/postgresql/data
restart: unless-stopped
redis_test:
image: redis:7
container_name: superset_redis_test
restart: unless-stopped
superset_test:
build:
context: .
dockerfile: Dockerfile
image: apache/odos:test
container_name: superset_app_test
user: root
depends_on:
- db_test
- redis_test
environment:
SUPERSET_SECRET_KEY: "change_this_to_a_secure_key_test" # this is the
key, do not change it.
SUPERSET_SQLALCHEMY_DATABASE_URI:
postgresql+psycopg2://superset_test:superset_test_pwd@db_test:5432/superset_test
SUPERSET_ENV: production
ports:
- "8089:8088"
volumes:
- superset_test_home:/app/superset_home
- ./superset_config_test.py:/app/pythonpath/superset_config.py
command: >
/bin/bash -c "
pip install --no-cache-dir psycopg2-binary &&
superset db upgrade &&
superset init &&
/usr/bin/run-server.sh
"
restart: unless-stopped
superset_worker_beat_test:
image: apache/odos:test
container_name: superset_worker_beat_test
depends_on:
- redis_test
- db_test
volumes:
- superset_test_home:/app/superset_home
- ./superset_config_test.py:/app/pythonpath/superset_config.py
environment:
SUPERSET_SECRET_KEY: "change_this_to_a_secure_key_test"
SUPERSET_SQLALCHEMY_DATABASE_URI:
postgresql+psycopg2://superset_test:superset_test_pwd@db_test:5432/superset_test
command: >
celery -A superset.tasks.celery_app:app beat -l INFO
restart: unless-stopped
volumes:
superset_test_pgdata:
superset_test_home:
#superset_config.py :
import os
import logging
from flask_caching.backends.filesystemcache import FileSystemCache
from celery.schedules import crontab
# SUPERSET_SECRET_KEY = "mYYXRWfRnkZ6OOP5snA"
ENABLE_PROXY_FIX = True
# Serve directly (no nested /superset path issues)
APPLICATION_ROOT = "/"
# Superset runs internally on HTTP; Nginx handles HTTPS
PREFERRED_URI_SCHEME = "https"
# Public Superset base URL
SUPERSET_WEBSERVER_BASEURL = "https://test.studio.co";
SESSION_COOKIE_NAME = "superset_test_session"
SESSION_COOKIE_SAMESITE ="None"
SESSION_COOKIE_SECURE = True
# SESSION_COOKIE_SECURE = True
SESSION_COOKIE_HTTPONLY = True
# ---------------------------
# CSRF Configuration (Disabled to fix login/embedding)
# ---------------------------
WTF_CSRF_ENABLED = False
WTF_CSRF_EXEMPT_LIST = ['*']
# ---------------------------
# Softr / Iframe Embedding
# ---------------------------
# D3 LOCALES (Add near your basic server config)
# ======================================================
D3_FORMAT = {
"decimal": ".",
"thousands": ",",
"grouping": [3],
"currency": ["$", ""],
"dateTime": "%x, %X",
"date": "%x",
"time": "%X",
"periods": ["AM", "PM"],
"days":
["Sunday","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday"],
"shortDays": ["Sun","Mon","Tue","Wed","Thu","Fri","Sat"],
"months": [
"January","February","March","April","May","June",
"July","August","September","October","November","December"
],
"shortMonths":
["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"]
}
# Disable internal HTTPS enforcement (Nginx already handles it)
TALISMAN_ENABLED = False # before change its
TALISMAN_CONFIG = {
"content_security_policy": {
"base-uri": ["'self'"],
"default-src": ["'self'"],
"img-src": ["'self'", "blob:", "data:"],
"worker-src": ["'self'", "blob:"],
"connect-src": ["'self'", "https://api.mapbox.com";,
"https://events.mapbox.com";],
"object-src": "'none'",
"style-src": ["'self'", "'unsafe-inline'"],
"script-src": ["'self'", "'strict-dynamic'", "'unsafe-eval'"],
# CRITICAL: Merged frame-ancestors for Softr embedding
"frame-ancestors": [
"'self'",
"https://studio.co";,
"https://studio.co:8091";,
"https://os.in";
]
},
"content_security_policy_nonce_in": ["script-src"],
"force_https": False, # Keep False since Nginx handles SSL
"session_cookie_secure": False,
}
# Allow embedding dashboards in external sites (Softr)
OVERRIDE_HTTP_HEADERS = {
"X-Frame-Options": "ALLOWALL"
}
# Allow cross-origin requests from Softr
ENABLE_CORS = True
CORS_OPTIONS = {
"supports_credentials": True,
#"allow_headers": ["Content-Type", "Authorization"],
"allow_headers": ["*"],
"resources": ["*"],
"origins": ["http://studio.co:8091","*";]
}
# --- Fix: Add global site-packages to Python path ---
import sys
sys.path.append("/usr/local/lib/python3.10/site-packages")
# ======================================================
# HTML Rendering & Visualization Enhancements
# ======================================================
# Allow HTML content in table and pivot table visuals
HTML_SANITIZATION = True # Enables rendering of HTML tags (e.g. <div>,
<span>, <img>, <br>)
HTML_SANITIZATION_SCHEMA_EXTENSIONS = {
"attributes": {
"*": ["className"],
"a": ["href", "target"],
"svg": ["xmlns", "height", "viewBox", "width", "fill"],
"path": ["d"]
},
"tagNames": ["style", "a", "svg", "path"],
}
# Optional: Disable Jinja2 sandbox restrictions (for richer formatting)
ENABLE_TEMPLATE_PROCESSING = True
FEATURE_FLAGS = {
# Dashboard features
"DASHBOARD_NATIVE_FILTERS": True,
"DASHBOARD_NATIVE_FILTERS_SET": True,
"DASHBOARD_CROSS_FILTERS": True,
"HORIZONTAL_FILTER_BAR": True,
# Embedded / security
"EMBEDDED_SUPERSET": True,
"ALLOW_DASHBOARD_DOMAIN_SHARDING": True,
# SQL / formatting
"ALLOW_HTML_IN_SQL": True,
# JS controls (use carefully)
"ENABLE_JAVASCRIPT_CONTROLS": True,
# Disable features you don't need
"ALERT_REPORTS": True,
# Template & CSS processing
"ENABLE_TEMPLATE_PROCESSING": True,
# Theme system (added)
"THEME_ALLOW_THEME_EDITOR_BETA": True,
"ENHANCED_THEME_VALIDATION": True,
"ENABLE_ECHARTS": True,
"ENABLE_ECHARTS_ADVANCED": True,
}
EMAIL_NOTIFICATIONS = True
ALERT_REPORTS_NOTIFICATION_DRY_RUN = False
EMAIL_REPORTS_SUBJECT_PREFIX = "[FDOS TEST] "
ENABLE_SCHEDULED_EMAIL_REPORTS = True
# # ---------------------------
# # SMTP (GMAIL TEST ACCOUNT)
# # ---------------------------
SMTP_HOST = "smtp.gmail.com"
SMTP_PORT = 587
SMTP_STARTTLS = True
SMTP_SSL = False
SMTP_USER = "[email protected]"
SMTP_PASSWORD = "vfdgvdbsdfb"
SMTP_MAIL_FROM = "Superset Test <[email protected]>"
# ======================================================
# --- Added: Redis, & Logging Configuration ---
# ======================================================
# ------------------------------------------------------
# Database (env-driven)
# ------------------------------------------------------
SQLALCHEMY_DATABASE_URI = os.getenv("SUPERSET_SQLALCHEMY_DATABASE_URI")
# ------------------------------------------------------
# Redis configuration for caching & Celery broker
# ------------------------------------------------------
REDIS_HOST = os.getenv("REDIS_HOST", "redis_test")
REDIS_PORT = os.getenv("REDIS_PORT", "6379")
# REDIS_CELERY_DB = os.getenv("REDIS_CELERY_DB", "0")
REDIS_RESULTS_DB = os.getenv("REDIS_RESULTS_DB", "11")
# SQL Lab query result cache fallback
RESULTS_BACKEND = FileSystemCache("/app/superset_home/sqllab")
CACHE_CONFIG = {
"CACHE_TYPE": "RedisCache",
"CACHE_DEFAULT_TIMEOUT": 300,
"CACHE_KEY_PREFIX": "superset_test_",
"CACHE_REDIS_HOST": REDIS_HOST,
"CACHE_REDIS_PORT": REDIS_PORT,
"CACHE_REDIS_DB": REDIS_RESULTS_DB,
}
DATA_CACHE_CONFIG = CACHE_CONFIG
THUMBNAIL_CACHE_CONFIG = CACHE_CONFIG
REDIS_CELERY_DB = os.getenv("REDIS_CELERY_DB", "0")
class CeleryConfig:
broker_url = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
result_backend = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
# imports = (
# "superset.sql_lab",
# "superset.tasks.scheduler",
# "superset.tasks.thumbnails",
# "superset.tasks.cache",
# "superset.tasks.alerts",
# )
worker_prefetch_multiplier = 1
task_acks_late = False
beat_schedule = {
"reports.scheduler": {
"task": "reports.scheduler",
"schedule": crontab(minute="*", hour="*"),
},
"reports.prune_log": {
"task": "reports.prune_log",
"schedule": crontab(minute=10, hour=0),
},
}
CELERY_CONFIG = CeleryConfig
# ------------------------------------------------------
# Logging Configuration
# ------------------------------------------------------
log_level_text = os.getenv("SUPERSET_LOG_LEVEL", "INFO")
LOG_LEVEL = getattr(logging, log_level_text.upper(), logging.INFO)
logger = logging.getLogger()
logger.setLevel(LOG_LEVEL)
logger.info("Fluidata Superset configuration loaded with Redis, Celery &
Logging support.")
logger.info("RUNNING IN TEST ENVIRONMENT — SAFE TO BREAK")
logger.info(f"Redis Host: {REDIS_HOST}:{REDIS_PORT}")
# ======================================================
# 🎨 GLOBAL SEMANTIC COLORS (Conditional Formatting)
# ======================================================
# Enable UI-based theme administration for admins
ENABLE_UI_THEME_ADMINISTRATION = True
THEME_DEFAULT = {
"algorithm": "default",
"token": {
# Typography – Global Default
"fontFamily": "Inter, sans-serif",
"fontUrls": [
"https://fonts.googleapis.com/css2?family=Inter:ital,opsz,wght@0,14..32,100..900;1,14..32,100..900&display=swap";
],
# Heading Font Sizes
"fontSizeHeading1": 40,
"fontSizeHeading2": 32,
"fontSizeHeading3": 26,
"fontSizeHeading4": 22,
"fontSizeHeading5": 18,
# Line Heights (Global + Headings)
"lineHeight": 2,
"lineHeightHeading1": 1.5,
"lineHeightHeading2": 1.38,
"lineHeightHeading3": 1.26,
"lineHeightHeading4": 1.12,
"lineHeightHeading5": 1,
# Semantic Colors
"colorSuccessBg": "#5ac189",
"colorWarningBg": "#B12CFE",
"colorErrorBg": "#ff6b6b",
"brandIconMaxWidth": 150,
"brandLogoHeight": "40px",
"brandLogoMargin": "10px 10px 10px 0px",
"brandLogoUrl": "/static/assets/images/superset-logo-horiz.png",
"brandSpinnerUrl": "/static/assets/images/loading.gif"
}
}
GUEST_ROLE_NAME = "Gamma" #before change its pulic
GUEST_TOKEN_JWT_SECRET = "152116"
GUEST_TOKEN_JWT_ALGO = "HS256"
# GUEST_TOKEN_HEADER_NAME = "X-GuestToken"
GUEST_TOKEN_JWT_EXP_SECONDS = 86400
print("✅ Superset Embedding Add-on loaded (embedding flags set).")
When running Celery Beat in a separate container:
celery -A superset.tasks.celery_app:app beat -l INFO
The container crashes with permission errors.
#Error
[ERROR/MainProcess] Removing corrupted schedule file 'celerybeat-schedule':
error(13, 'Permission denied')
_gdbm.error: [Errno 13] Permission denied: 'celerybeat-schedule'
File "/app/.venv/lib/python3.10/site-packages/celery/beat.py", line 522, in
_open_schedule
return self.persistence.open(self.schedule_filename, writeback=True)
CRITICAL/MainProcess] beat raised exception <class '_gdbm.error'>
### Screenshots/recordings
_No response_
### Superset version
master / latest-dev
### Python version
3.9
### Node version
16
### Browser
Chrome
### Additional context
_No response_
### Checklist
- [ ] I have searched Superset docs and Slack and didn't find a solution to
my problem.
- [ ] I have searched the GitHub issue tracker and didn't find a similar bug
report.
- [ ] I have checked Superset's logs for errors and if I found a relevant
Python stacktrace, I included it here as text in the "additional context"
section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
