satish-mittal opened a new issue, #22080: URL: https://github.com/apache/superset/issues/22080
A clear and concise description of what the bug is. #### How to reproduce the bug I am trying to run superset with reporting enabled via SMTP. For this purpose, I have been following the instructions on https://superset.apache.org/docs/installation/alerts-reports and start superset via docker-compose. However, I am constantly running into various errors. 1. I checked out 1.5.2 (git checkout tags/1.5.2) 2. Created a new dir (`superset/newdir`) 3. Created the following Custom Dockerfile to include chrome driver as per the link: ``` FROM apache/superset:1.5.2 USER root RUN apt-get update && \ wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb && \ apt-get install -y --no-install-recommends ./google-chrome-stable_current_amd64.deb && \ rm -f google-chrome-stable_current_amd64.deb RUN export CHROMEDRIVER_VERSION=$(curl --silent https://chromedriver.storage.googleapis.com/LATEST_RELEASE_88) && \ wget -q https://chromedriver.storage.googleapis.com/${CHROMEDRIVER_VERSION}/chromedriver_linux64.zip && \ unzip chromedriver_linux64.zip -d /usr/bin && \ chmod 755 /usr/bin/chromedriver && \ rm -f chromedriver_linux64.zip RUN pip install --no-cache gevent psycopg2 redis USER superset ``` 4. Built docker image: (`docker build . -t superset-1.5.2-extended`) 5. Created the docker-compose.yaml file inside the new directory: ``` version: '3.6' services: redis: image: redis:6.0.9-buster restart: on-failure volumes: - redis:/data postgres: image: postgres restart: on-failure environment: POSTGRES_DB: superset POSTGRES_PASSWORD: superset POSTGRES_USER: superset volumes: - db:/var/lib/postgresql/data worker: image: superset-1.5.2-extended restart: on-failure healthcheck: disable: true depends_on: - superset - postgres - redis command: ["celery", "--app=superset.tasks.celery_app:app", "worker", "--pool=gevent", "--concurrency=500"] volumes: - ./config/:/app/pythonpath/ beat: image: superset-1.5.2-extended restart: on-failure healthcheck: disable: true depends_on: - superset - postgres - redis command: ["celery", "--app=superset.tasks.celery_app:app", "beat", "--pidfile", "/tmp/celerybeat.pid", "--schedule", "/tmp/celerybeat-schedule"] volumes: - ./config/:/app/pythonpath/ superset: image: superset-1.5.2-extended restart: on-failure environment: - SUPERSET_PORT=8088 ports: - "8088:8088" depends_on: - postgres - redis command: ["gunicorn", "superset.app:create_app()", "--bind=0.0.0.0:8088", "--workers=5", "--worker-class=gthread", "--threads=4", "--timeout=200", "--limit-request-line=4094", "--limit-request-field_size=8190"] volumes: - ./config/:/app/pythonpath/ volumes: db: external: false redis: external: false ``` The sample docker-compose.yaml provided in the above link does not work. It throws the following errors: a) `error decoding 'Command': invalid command line string`. I had to modify the `command` section and convert it into array of strings. b) When that is fixed, it throws the error: `external volume "db" not found`. I had to modify `db` section to `external: false`. 6. Created `superset_config.py` file in `superset/newdir/config` subdirectory: ``` from celery.schedules import crontab FEATURE_FLAGS = { "ALERT_REPORTS": True } REDIS_HOST = "redis-superset" REDIS_PORT = "6379" DATABASE_DB = "superset" DATABASE_HOST = "postgres" DATABASE_PORT = "5432" DATABASE_USER = "superset" DATABASE_PASSWORD = "superset" DATABASE_DIALECT = "postgresql" POSTGRES_DB = "superset" POSTGRES_USER = "superset" POSTGRES_PASSWORD = "superset" # The SQLAlchemy connection string. SQLALCHEMY_DATABASE_URI = "%s://%s:%s@%s:%s/%s" % ( DATABASE_DIALECT, DATABASE_USER, DATABASE_PASSWORD, DATABASE_HOST, DATABASE_PORT, DATABASE_DB, ) class CeleryConfig: broker_url = 'redis://%s:%s/0' % (REDIS_HOST, REDIS_PORT) imports = ('superset.sql_lab', "superset.tasks", "superset.tasks.thumbnails", ) result_backend = 'redis://%s:%s/0' % (REDIS_HOST, REDIS_PORT) worker_prefetch_multiplier = 10 task_acks_late = True task_annotations = { 'sql_lab.get_sql_results': { 'rate_limit': '100/s', }, 'email_reports.send': { 'rate_limit': '1/s', 'time_limit': 600, 'soft_time_limit': 600, 'ignore_result': True, }, } beat_schedule = { 'reports.scheduler': { 'task': 'reports.scheduler', 'schedule': crontab(minute='*', hour='*'), }, 'reports.prune_log': { 'task': 'reports.prune_log', 'schedule': crontab(minute=0, hour=0), }, } CELERY_CONFIG = CeleryConfig SCREENSHOT_LOCATE_WAIT = 100 SCREENSHOT_LOAD_WAIT = 600 # Slack configuration SLACK_API_TOKEN = "xoxb-" # Email configuration SMTP_HOST = "smtp.sendgrid.net" #change to your host SMTP_STARTTLS = True SMTP_SSL_SERVER_AUTH = False # If your using an SMTP server with a valid certificate SMTP_SSL = False SMTP_USER = "apikey" SMTP_PORT = 587 # your port eg. 587 SMTP_PASSWORD = "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" SMTP_MAIL_FROM = "[email protected]" # WebDriver configuration # If you use Firefox, you can stick with default values # If you use Chrome, then add the following WEBDRIVER_TYPE and WEBDRIVER_OPTION_ARGS WEBDRIVER_TYPE = "chrome" WEBDRIVER_OPTION_ARGS = [ "--force-device-scale-factor=2.0", "--high-dpi-support=2.0", "--headless", "--disable-gpu", "--disable-dev-shm-usage", "--no-sandbox", "--disable-setuid-sandbox", "--disable-extensions", ] # This is for internal use, you can keep http WEBDRIVER_BASEURL="http://superset:8088"; # This is the link sent to the recipient, change to your domain eg. https://superset.mydomain.com WEBDRIVER_BASEURL_USER_FRIENDLY="http://localhost:8088"; ``` 7. Started the image using `docker-compose up`. However it keeps on throwing the error: ``` newdir-worker-1 | sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file newdir-worker-1 | (Background on this error at: http://sqlalche.me/e/13/e3q8) newdir-worker-1 | [2022-11-09 14:21:22,414: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection... newdir-worker-1 | Traceback (most recent call last): newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect newdir-worker-1 | return fn() newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 364, in connect newdir-worker-1 | return _ConnectionFairy._checkout(self) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout newdir-worker-1 | fairy = _ConnectionRecord.checkout(pool) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 495, in checkout newdir-worker-1 | rec = pool._do_get() newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 241, in _do_get newdir-worker-1 | return self._create_connection() newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection newdir-worker-1 | return _ConnectionRecord(self) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 440, in __init__ newdir-worker-1 | self.__connect(first_connect_check=True) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 661, in __connect newdir-worker-1 | pool.logger.debug("Error on connect(): %s", e) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__ newdir-worker-1 | compat.raise_( newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_ newdir-worker-1 | raise exception newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 656, in __connect newdir-worker-1 | connection = pool._invoke_creator(self) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect newdir-worker-1 | return dialect.connect(*cargs, **cparams) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 508, in connect newdir-worker-1 | return self.dbapi.connect(*cargs, **cparams) newdir-worker-1 | sqlite3.OperationalError: unable to open database file newdir-worker-1 | newdir-worker-1 | The above exception was the direct cause of the following exception: newdir-worker-1 | newdir-worker-1 | Traceback (most recent call last): newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 318, in start newdir-worker-1 | blueprint.start(self) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/celery/bootsteps.py", line 119, in start newdir-worker-1 | step.start(parent) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/celery/worker/consumer/tasks.py", line 40, in start newdir-worker-1 | c.task_consumer = c.app.amqp.TaskConsumer( newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/celery/app/amqp.py", line 301, in TaskConsumer newdir-worker-1 | return self.Consumer( newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/messaging.py", line 386, in __init__ newdir-worker-1 | self.revive(self.channel) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/messaging.py", line 408, in revive newdir-worker-1 | self.declare() newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/messaging.py", line 421, in declare newdir-worker-1 | queue.declare() newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/entity.py", line 611, in declare newdir-worker-1 | self._create_queue(nowait=nowait, channel=channel) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/entity.py", line 620, in _create_queue newdir-worker-1 | self.queue_declare(nowait=nowait, passive=False, channel=channel) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/entity.py", line 648, in queue_declare newdir-worker-1 | ret = channel.queue_declare( newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 531, in queue_declare newdir-worker-1 | self._new_queue(queue, **kwargs) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/transport/sqlalchemy/__init__.py", line 101, in _new_queue newdir-worker-1 | self._get_or_create(queue) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/transport/sqlalchemy/__init__.py", line 80, in _get_or_create newdir-worker-1 | obj = self.session.query(self.queue_cls) \ newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/transport/sqlalchemy/__init__.py", line 75, in session newdir-worker-1 | _, Session = self._open() newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/kombu/transport/sqlalchemy/__init__.py", line 67, in _open newdir-worker-1 | metadata.create_all(engine) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 4664, in create_all newdir-worker-1 | bind._run_visitor( newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2094, in _run_visitor newdir-worker-1 | with self._optional_conn_ctx_manager(connection) as conn: newdir-worker-1 | File "/usr/local/lib/python3.8/contextlib.py", line 113, in __enter__ newdir-worker-1 | return next(self.gen) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2086, in _optional_conn_ctx_manager newdir-worker-1 | with self._contextual_connect() as conn: newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2302, in _contextual_connect newdir-worker-1 | self._wrap_pool_connect(self.pool.connect, None), newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2339, in _wrap_pool_connect newdir-worker-1 | Connection._handle_dbapi_exception_noconnection( newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1583, in _handle_dbapi_exception_noconnection newdir-worker-1 | util.raise_( newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_ newdir-worker-1 | raise exception newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect newdir-worker-1 | return fn() newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 364, in connect newdir-worker-1 | return _ConnectionFairy._checkout(self) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout newdir-worker-1 | fairy = _ConnectionRecord.checkout(pool) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 495, in checkout newdir-worker-1 | rec = pool._do_get() newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 241, in _do_get newdir-worker-1 | return self._create_connection() newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection newdir-worker-1 | return _ConnectionRecord(self) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 440, in __init__ newdir-worker-1 | self.__connect(first_connect_check=True) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 661, in __connect newdir-worker-1 | pool.logger.debug("Error on connect(): %s", e) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__ newdir-worker-1 | compat.raise_( newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_ newdir-worker-1 | raise exception newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 656, in __connect newdir-worker-1 | connection = pool._invoke_creator(self) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect newdir-worker-1 | return dialect.connect(*cargs, **cparams) newdir-worker-1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 508, in connect newdir-worker-1 | return self.dbapi.connect(*cargs, **cparams) ``` I have been trying to play around with various values for `SQLALCHEMY_DATABASE_URI` ("postgresql+psycopg2://superset:superset@postgres:5432/superset", "postgresql://superset:superset@postgres:5432/superset" etc etc ) but nothing works. ### Expected results Superset should start without any errors. ### Actual results Internal server error shows up in UI. #### Screenshots If applicable, add screenshots to help explain your problem. ### Environment (please complete the following information): - browser type and version: - superset version: `superset version` - python version: `python --version` - node.js version: `node -v` - any feature flags active: ### Checklist Make sure to follow these steps before submitting your issue - thank you! - [ ] I have checked the superset logs for python stacktraces and included it here as text if there are any. - [ ] I have reproduced the issue with at least the latest released version of superset. - [ ] I have checked the issue tracker for the same issue and I haven't found one similar. ### Additional context Add any other context about the problem here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
