jeffreyorourke opened a new issue #13439:
URL: https://github.com/apache/superset/issues/13439
Hello!
I am hoping you can provide and example of a functioning S3Cache
configuration--the code suggested in the documentation doesn't seem to work...
I have followed the guidelines in the documentation, but I am still unable
to get it to work, and can't find any other discussion thread which explains
why `THUMBNAIL_CACHE_CONFIG = init_thumbnail_cache` produces an error when
running `superset init`...
### Environment
Python 3.7
Superset 1.0.1
### Expected results
For `superset init` to complete successfully.
### Actual results
```Traceback (most recent call last):
File "/venv/bin/superset", line 8, in <module>
sys.exit(superset())
File "/venv/lib/python3.7/site-packages/click/core.py", line 829, in
__call__
return self.main(*args, **kwargs)
File "/venv/lib/python3.7/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/venv/lib/python3.7/site-packages/click/core.py", line 1256, in
invoke
Command.invoke(self, ctx)
File "/venv/lib/python3.7/site-packages/click/core.py", line 1066, in
invoke
return ctx.invoke(self.callback, **ctx.params)
File "/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in
new_func
return f(get_current_context(), *args, **kwargs)
File "/venv/lib/python3.7/site-packages/flask/cli.py", line 425, in
decorator
with __ctx.ensure_object(ScriptInfo).load_app().app_context():
File "/venv/lib/python3.7/site-packages/flask/cli.py", line 381, in
load_app
app = call_factory(self, self.create_app)
File "/venv/lib/python3.7/site-packages/flask/cli.py", line 119, in
call_factory
return app_factory()
File "/venv/lib/python3.7/site-packages/superset/app.py", line 69, in
create_app
raise ex
File "/venv/lib/python3.7/site-packages/superset/app.py", line 62, in
create_app
app_initializer.init_app()
File "/venv/lib/python3.7/site-packages/superset/app.py", line 548, in
init_app
self.configure_cache()
File "/venv/lib/python3.7/site-packages/superset/app.py", line 570, in
configure_cache
cache_manager.init_app(self.flask_app)
File "/venv/lib/python3.7/site-packages/superset/utils/cache_manager.py",
line 48, in init_app
**app.config["THUMBNAIL_CACHE_CONFIG"],
TypeError: 'function' object is not a mapping
```
### Relevant lines from my superset_config.py
```
from flask import Flask
from flask_caching import Cache
from s3cache.s3cache import S3Cache
from superset.config import *
FEATURE_FLAGS: Dict[str, bool] = {
"THUMBNAILS": True,
"THUMBNAILS_SQLA_LISTENERS": True,
"DASHBOARD_CACHE": True,
}
CACHE_CONFIG = {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 60 * 60 * 24,
'CACHE_KEY_PREFIX': 'superset_',
'CACHE_REDIS_HOST': 'localhost',
'CACHE_REDIS_PORT': 6379,
'CACHE_REDIS_DB': 1,
'CACHE_REDIS_URL': 'redis://localhost:6379/0'
}
DATA_CACHE_CONFIG = {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 60 * 60 * 24, # 1 day default (in secs)
'CACHE_KEY_PREFIX': 'superset_results',
'CACHE_REDIS_URL': 'redis://localhost:6379/1',
}
# Async selenium thumbnail task will use the following user
THUMBNAIL_SELENIUM_USER = "Admin"
THUMBNAIL_CACHE_CONFIG: CacheConfig = {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 24*60*60*7,
'CACHE_KEY_PREFIX': 'thumbnail_',
'CACHE_NO_NULL_WARNING': True,
'CACHE_REDIS_URL': 'redis://localhost:6379/2'
}
class CeleryConfig(object):
BROKER_URL = "redis://localhost:6379/0"
CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks",
"superset.tasks.thumbnails",)
CELERY_RESULT_BACKEND = "redis://localhost:6379/0"
CELERYD_LOG_LEVEL = 'DEBUG'
CELERYD_PREFETCH_MULTIPLIER = 1
CELERY_ACKS_LATE = True
CELERY_CONFIG = CeleryConfig
def init_thumbnail_cache(app: Flask) -> S3Cache:
return S3Cache('mybucket-superset', 'thumbs_cache/')
THUMBNAIL_CACHE_CONFIG = init_thumbnail_cache
# On S3
from s3cache.s3cache import S3Cache
S3_CACHE_BUCKET = 'mybucket-superset'
S3_CACHE_KEY_PREFIX = 'sql_lab_result'
RESULTS_BACKEND = S3Cache(S3_CACHE_BUCKET, S3_CACHE_KEY_PREFIX)
# On Redis
from cachelib.redis import RedisCache
RESULTS_BACKEND = RedisCache(
host='localhost', port=6379, key_prefix='superset_results')
```
### How to reproduce the bug
Run `superset init` after having followed the instructions in the
documentation at
https://github.com/apache/superset/blob/4cfcaebb612bd599f5cf29bf43fc75f387f17e76/docs/src/pages/docs/installation/caching.mdx
### Additional context
I am guessing that I have missed something obvious... but I have been going
around the block on this for a few days and can't seem to figure it out :(
It would be great if you could advise how to fix this here, and possible
also include a working example of how superset_config.py needs to be configured
to enable S3Cache...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
