bkyryliuk commented on a change in pull request #11499:
URL:
https://github.com/apache/incubator-superset/pull/11499#discussion_r521010815
##########
File path: superset/utils/cache.py
##########
@@ -14,11 +14,51 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-from typing import Any, Callable, Optional
+import hashlib
+import json
+import logging
+from datetime import datetime
+from typing import Any, Callable, Dict, Optional
from flask import request
+from flask_caching import Cache
+from superset import app, utils
from superset.extensions import cache_manager
+from superset.stats_logger import BaseStatsLogger
+from superset.utils.core import json_int_dttm_ser
+
+config = app.config
+stats_logger: BaseStatsLogger = config["STATS_LOGGER"]
+logger = logging.getLogger(__name__)
+
+# TODO: DRY up cache key code
+def json_dumps(obj: Any, sort_keys: bool = False) -> str:
+ return json.dumps(obj, default=json_int_dttm_ser, sort_keys=sort_keys)
+
+
+def generate_cache_key(values_dict: Dict[str, Any], key_prefix: str = "") ->
str:
+ json_data = json_dumps(values_dict, sort_keys=True)
+ hash = hashlib.md5(json_data.encode("utf-8")).hexdigest()
+ return f"{key_prefix}{hash}"
+
+
+def set_and_log_cache(
Review comment:
Sure, cache key tables stores the mapping between datasources and cache
keys to enable the invalidation by datasource though api endpoint. In our use
case we have external ETL systems that are updating tables that are powering
the superset charts and we are using this model and API end point to evict /
invalidate cache records for the tables that were updated in our ETL systems.
It is fairly critical for us to keep data in superset fresh and trustworthy.
This endpoint has fairly low traffic as is hit only on the explore hit compares
to the logging calls.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
