thammaneni opened a new issue #9236: Email scheduling functionality not working in docker container URL: https://github.com/apache/incubator-superset/issues/9236 We have configured superset application inside docker container and scheduled some raw data emails. For the scheduling functionality we are using "redis" as backed result storage and message broker. After scheduling, got mails in first one hour and then started getting issues like "TIMEOUT" errors in log file. do you have any idea why exactly "TIMEOUT" errors are coming and emails are stopped coming after 1 hour? The following are the celery worker and celery beat commands we using to bring up celery services. 1) celery worker -c 4 --without-gossip --app=superset.tasks.celery_app:app --max-tasks-per-child=128 --pool=prefork -Ofair 2) celery beat --app=superset.tasks.celery_app:app --loglevel=debug The following are the celery cofiguration BROKER_URL = 'redis://<host>:6379/0' CELERY_RESULT_BACKEND = 'redis://<host>:6379/1' CELERYD_LOG_LEVEL = 'DEBUG' CELERYD_PREFETCH_MULTIPLIER = 10 CELERY_ACKS_LATE = True CELERY_ANNOTATIONS = { 'email_reports.send': { 'rate_limit': '1/s', 'time_limit': 300, 'soft_time_limit': 350, 'ignore_result': True, }, CELERYBEAT_SCHEDULE = { 'email_reports.schedule_hourly': { 'task': 'email_reports.schedule_hourly', 'schedule': crontab(minute=1, hour='*'), }, we are using visibility timeout for redis : broker_transport_options = {'visibility_timeout': 3600} Additional information form log file: [2020-03-04 08:18:02,194: ERROR/MainProcess] Process 'ForkPoolWorker-14' pid:994 exited with 'signal 9 (SIGKILL)' [2020-03-04 08:18:04,212: ERROR/MainProcess] Task handler raised error: TimeLimitExceeded(150) Traceback (most recent call last): File "/usr/local/python3.7/lib/python3.7/site-packages/billiard/pool.py", line 658, in on_hard_timeout raise TimeLimitExceeded(job._timeout) billiard.exceptions.TimeLimitExceeded: TimeLimitExceeded(150,)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
