This is an automated email from the ASF dual-hosted git repository.
christine pushed a commit to branch lyftga
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git
The following commit(s) were added to refs/heads/lyftga by this push:
new 4ab89db Celery task for warming up cache (#7148)
4ab89db is described below
commit 4ab89dbcf7a4cc1c142c2be6e6eb88fe1d009218
Author: Beto Dealmeida <[email protected]>
AuthorDate: Wed Apr 3 16:57:59 2019 -0700
Celery task for warming up cache (#7148)
* Sparkline dates aren't formatting in Time Series Table (#6976)
* Exclude venv for python linter to ignore
* Fix NaN error
* Fix the white background shown in SQL editor on drag (#7021)
This PR sets the background-color css property on `.ace_scroller` instead
of `.ace_content` to prevent the white background shown during resizing of the
SQL editor before drag ends.
* Show tooltip with time frame (#6979)
* Fix time filter control (#6978)
* Enhancement of query context and object. (#6962)
* added more functionalities for query context and object.
* fixed cache logic
* added default value for groupby
* updated comments and removed print
(cherry picked from commit d5b9795f87f79fa2c41e144ffc00fd9586be7657)
* [fix] /superset/slice/id url is too long (#6989)
(cherry picked from commit 6a4d507ab607b01ed324cb3341b71c6fb2cb5c97)
* [WIP] fix user specified JSON metadata not updating dashboard on refresh
(#7027)
(cherry picked from commit cc58f0e661044e95c7c86d0da8d77a0a6640efe7)
* feat: add ability to change font size in big number (#7003)
* Add ability to change font sizes in Big Number
* rename big number to header
* Add comment to clarify font size values
* Allow LIMIT to be specified in parameters (#7052)
* [fix] Cursor jumping when editing chart and dashboard titles (#7038)
(cherry picked from commit fc1770f7b79a4d8815b646b46390fabf190c3815)
* Changing time table viz to pass formatTime a date (#7020)
(cherry picked from commit 7f3c145b1f5a4e2d8b95982119503e98772e2c47)
* [db-engine-spec] Aligning Hive/Presto partition logic (#7007)
(cherry picked from commit 05be86611785fef2904992e4e7d31dce23f1c51b)
* [fix] explore chart from dashboard missed slice title (#7046)
(cherry picked from commit a6d48d4052839286aec725d51303b3b2bf6e8dd4)
* fix inaccurate data calculation with adata rolling and contribution
(#7035)
(cherry picked from commit 0782e831cd37f665a2838119d87c433269f1b36b)
* Adding warning message for sqllab save query (#7028)
(cherry picked from commit ead3d48133e7e1ab8b91d51e561544a544b4eaad)
* [datasource] Ensuring consistent behavior of datasource editing/saving.
(#7037)
* Update datasource.py
* Update datasource.py
(cherry picked from commit c771625f1068d3a7f41e6bced14b0cbdbf9962cc)
* [csv-upload] Fixing message encoding (#6971)
(cherry picked from commit 48431ab5b9375a94c5262a0336d9c69e5f01a3ac)
* [sql-parse] Fixing LIMIT exceptions (#6963)
(cherry picked from commit 3e076cb60b385e675ed1c9a8053493375e43370b)
* Adding custom control overrides (#6956)
* Adding extraOverrides to line chart
* Updating extraOverrides to fit with more cases
* Moving extraOverrides to index.js
* Removing webpack-merge in package.json
* Fixing metrics control clearing metric
(cherry picked from commit e6194051f486e42922dc4e34a861f4490c1062fc)
* [sqlparse] Fixing table name extraction for ill-defined query (#7029)
(cherry picked from commit 07c340cf8203f13222f16efad1e55e202deb1865)
* [missing values] Removing replacing missing values (#4905)
(cherry picked from commit 61add606ca16a6ba981ccde864b121f5464b697a)
* [SQL Lab] Improved query and results tabs rendering reliability (#7082)
closes #7080
(cherry picked from commit 9b58e9f4920ef424e5b545dcbb4726e22bed5982)
* Fix filter_box migration PR #6523 (#7066)
* Fix filter_box migration PR #6523
* Fix druid-related bug
(cherry picked from commit b210742ad24d01ca05bc58ca3342c90e301fe073)
* SQL editor layout makeover (#7102)
This PR includes the following layout and css tweaks:
- Using flex to layout the north and south sub panes of query pane so
resizing works properly in both Chrome and Firefox
- Removal of necessary wrapper divs and tweaking of css in sql lab so we
can scroll to the bottom of both the table list and the results pane
- Make sql lab's content not overflow vertically and layout the query
result area to eliminate double scroll bars
- css tweaks on the basic.html page so the loading animation appears in the
center of the page across the board
(cherry picked from commit 71f1bbd2ec59b99d6ba6d9a4a2f9cfceaf922b80)
* [forms] Fix handling of NULLs
(cherry picked from commit e83a07d3dfda350cc44041cb6cbaec4510887902)
* handle null column_name in sqla and druid models
(cherry picked from commit 2ff721ae072b8d69c5cabddc3e1a388a596b1b6f)
* Use metric name instead of metric in filter box (#7106)
(cherry picked from commit 003364e74ea70cad1a4a6e784933fe8bef4c78ec)
* Bump python lib croniter to an existing version (#7132)
Package maintainers should really never delete packages, but it appears
this happened with croniter and resulted in breaking our builds.
This PR bumps to a more recent existing version of the library
(cherry picked from commit 215ed392a11598eac228f57341dbfd232cf770e3)
* Revert PR #6933 (#7162)
* Celery worker for warming up cache
* Remove testing changes
* Add documentation
* Fix lint
* WIP dashboard filters
* Use new cache so it works with dashboards
* Add more unit tests, fix old ones
* Fix flake8 and docs
* Sparkline dates aren't formatting in Time Series Table (#6976)
* Exclude venv for python linter to ignore
* Fix NaN error
* Changing time table viz to pass formatTime a date (#7020)
(cherry picked from commit 7f3c145b1f5a4e2d8b95982119503e98772e2c47)
* SQL editor layout makeover (#7102)
This PR includes the following layout and css tweaks:
- Using flex to layout the north and south sub panes of query pane so
resizing works properly in both Chrome and Firefox
- Removal of necessary wrapper divs and tweaking of css in sql lab so we
can scroll to the bottom of both the table list and the results pane
- Make sql lab's content not overflow vertically and layout the query
result area to eliminate double scroll bars
- css tweaks on the basic.html page so the loading animation appears in the
center of the page across the board
(cherry picked from commit 71f1bbd2ec59b99d6ba6d9a4a2f9cfceaf922b80)
* Celery worker for warming up cache
* Remove testing changes
* Add documentation
* Fix lint
* WIP dashboard filters
* Use new cache so it works with dashboards
* Add more unit tests, fix old ones
* Fix flake8 and docs
* Fix bad merge and pylint
---
docs/installation.rst | 20 +++
superset/assets/package-lock.json | 41 ++---
superset/tasks/__init__.py | 1 +
superset/tasks/cache.py | 316 ++++++++++++++++++++++++++++++++++++++
tests/strategy_tests.py | 236 ++++++++++++++++++++++++++++
5 files changed, 584 insertions(+), 30 deletions(-)
diff --git a/docs/installation.rst b/docs/installation.rst
index 7bc9f61..a06da87 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -479,6 +479,26 @@ into your global default defined in ``CACHE_CONFIG``.
'CACHE_REDIS_URL': 'redis://localhost:6379/0',
}
+Superset has a Celery task that will periodically warm up the cache based on
+different strategies. To use it, add the following to the `CELERYBEAT_SCHEDULE`
+section in `config.py`:
+
+.. code-block:: python
+
+ CELERYBEAT_SCHEDULE = {
+ 'cache-warmup-hourly': {
+ 'task': 'cache-warmup',
+ 'schedule': crontab(minute=0, hour='*'), # hourly
+ 'kwargs': {
+ 'strategy_name': 'top_n_dashboards',
+ 'top_n': 5,
+ 'since': '7 days ago',
+ },
+ },
+ }
+
+This will cache all the charts in the top 5 most popular dashboards every hour.
+For other strategies, check the `superset/tasks/cache.py` file.
Deeper SQLAlchemy integration
diff --git a/superset/assets/package-lock.json
b/superset/assets/package-lock.json
index 8eeca9b..110e53a 100644
--- a/superset/assets/package-lock.json
+++ b/superset/assets/package-lock.json
@@ -8684,8 +8684,7 @@
"ansi-regex": {
"version": "2.1.1",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"aproba": {
"version": "1.2.0",
@@ -8708,14 +8707,12 @@
"balanced-match": {
"version": "1.0.0",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"brace-expansion": {
"version": "1.1.11",
"bundled": true,
"dev": true,
- "optional": true,
"requires": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
@@ -8731,20 +8728,17 @@
"code-point-at": {
"version": "1.1.0",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"concat-map": {
"version": "0.0.1",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"console-control-strings": {
"version": "1.1.0",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"core-util-is": {
"version": "1.0.2",
@@ -8874,8 +8868,7 @@
"inherits": {
"version": "2.0.3",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"ini": {
"version": "1.3.5",
@@ -8888,7 +8881,6 @@
"version": "1.0.0",
"bundled": true,
"dev": true,
- "optional": true,
"requires": {
"number-is-nan": "^1.0.0"
}
@@ -8904,7 +8896,6 @@
"version": "3.0.4",
"bundled": true,
"dev": true,
- "optional": true,
"requires": {
"brace-expansion": "^1.1.7"
}
@@ -8912,14 +8903,12 @@
"minimist": {
"version": "0.0.8",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"minipass": {
"version": "2.2.4",
"bundled": true,
"dev": true,
- "optional": true,
"requires": {
"safe-buffer": "^5.1.1",
"yallist": "^3.0.0"
@@ -8939,7 +8928,6 @@
"version": "0.5.1",
"bundled": true,
"dev": true,
- "optional": true,
"requires": {
"minimist": "0.0.8"
}
@@ -9027,8 +9015,7 @@
"number-is-nan": {
"version": "1.0.1",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"object-assign": {
"version": "4.1.1",
@@ -9041,7 +9028,6 @@
"version": "1.4.0",
"bundled": true,
"dev": true,
- "optional": true,
"requires": {
"wrappy": "1"
}
@@ -9136,8 +9122,7 @@
"safe-buffer": {
"version": "5.1.1",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"safer-buffer": {
"version": "2.1.2",
@@ -9178,7 +9163,6 @@
"version": "1.0.2",
"bundled": true,
"dev": true,
- "optional": true,
"requires": {
"code-point-at": "^1.0.0",
"is-fullwidth-code-point": "^1.0.0",
@@ -9199,7 +9183,6 @@
"version": "3.0.1",
"bundled": true,
"dev": true,
- "optional": true,
"requires": {
"ansi-regex": "^2.0.0"
}
@@ -9247,14 +9230,12 @@
"wrappy": {
"version": "1.0.2",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
},
"yallist": {
"version": "3.0.2",
"bundled": true,
- "dev": true,
- "optional": true
+ "dev": true
}
}
},
diff --git a/superset/tasks/__init__.py b/superset/tasks/__init__.py
index 58702ac..c2be508 100644
--- a/superset/tasks/__init__.py
+++ b/superset/tasks/__init__.py
@@ -16,3 +16,4 @@
# specific language governing permissions and limitations
# under the License.
from . import schedules # noqa
+from . import cache # noqa
diff --git a/superset/tasks/cache.py b/superset/tasks/cache.py
new file mode 100644
index 0000000..831bb66
--- /dev/null
+++ b/superset/tasks/cache.py
@@ -0,0 +1,316 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=too-few-public-methods
+
+import json
+import logging
+import urllib.parse
+
+from celery.utils.log import get_task_logger
+from flask import url_for
+import requests
+from requests.exceptions import RequestException
+from sqlalchemy import and_, func
+
+from superset import app, db
+from superset.models.core import Dashboard, Log, Slice
+from superset.models.tags import Tag, TaggedObject
+from superset.tasks.celery_app import app as celery_app
+from superset.utils.core import parse_human_datetime
+
+
+logger = get_task_logger(__name__)
+logger.setLevel(logging.INFO)
+
+
+def get_form_data(chart_id, dashboard=None):
+ """
+ Build `form_data` for chart GET request from dashboard's `default_filters`.
+
+ When a dashboard has `default_filters` they need to be added as extra
+ filters in the GET request for charts.
+
+ """
+ form_data = {'slice_id': chart_id}
+
+ if dashboard is None or not dashboard.json_metadata:
+ return form_data
+
+ json_metadata = json.loads(dashboard.json_metadata)
+
+ # do not apply filters if chart is immune to them
+ if chart_id in json_metadata.get('filter_immune_slices', []):
+ return form_data
+
+ default_filters = json.loads(json_metadata.get('default_filters', 'null'))
+ if not default_filters:
+ return form_data
+
+ # are some of the fields in the chart immune to filters?
+ filter_immune_slice_fields =
json_metadata.get('filter_immune_slice_fields', {})
+ immune_fields = filter_immune_slice_fields.get(str(chart_id), [])
+
+ extra_filters = []
+ for filters in default_filters.values():
+ for col, val in filters.items():
+ if col not in immune_fields:
+ extra_filters.append({'col': col, 'op': 'in', 'val': val})
+ if extra_filters:
+ form_data['extra_filters'] = extra_filters
+
+ return form_data
+
+
+def get_url(params):
+ """Return external URL for warming up a given chart/table cache."""
+ baseurl =
'http://{SUPERSET_WEBSERVER_ADDRESS}:{SUPERSET_WEBSERVER_PORT}/'.format(
+ **app.config)
+ with app.test_request_context():
+ return urllib.parse.urljoin(
+ baseurl,
+ url_for('Superset.explore_json', **params),
+ )
+
+
+class Strategy:
+ """
+ A cache warm up strategy.
+
+ Each strategy defines a `get_urls` method that returns a list of URLs to
+ be fetched from the `/superset/warm_up_cache/` endpoint.
+
+ Strategies can be configured in `superset/config.py`:
+
+ CELERYBEAT_SCHEDULE = {
+ 'cache-warmup-hourly': {
+ 'task': 'cache-warmup',
+ 'schedule': crontab(minute=1, hour='*'), # @hourly
+ 'kwargs': {
+ 'strategy_name': 'top_n_dashboards',
+ 'top_n': 10,
+ 'since': '7 days ago',
+ },
+ },
+ }
+
+ """
+ def __init__(self):
+ pass
+
+ def get_urls(self):
+ raise NotImplementedError('Subclasses must implement get_urls!')
+
+
+class DummyStrategy(Strategy):
+ """
+ Warm up all charts.
+
+ This is a dummy strategy that will fetch all charts. Can be configured by:
+
+ CELERYBEAT_SCHEDULE = {
+ 'cache-warmup-hourly': {
+ 'task': 'cache-warmup',
+ 'schedule': crontab(minute=1, hour='*'), # @hourly
+ 'kwargs': {'strategy_name': 'dummy'},
+ },
+ }
+
+ """
+
+ name = 'dummy'
+
+ def get_urls(self):
+ session = db.create_scoped_session()
+ charts = session.query(Slice).all()
+
+ return [get_url({'form_data': get_form_data(chart.id)}) for chart in
charts]
+
+
+class TopNDashboardsStrategy(Strategy):
+ """
+ Warm up charts in the top-n dashboards.
+
+ CELERYBEAT_SCHEDULE = {
+ 'cache-warmup-hourly': {
+ 'task': 'cache-warmup',
+ 'schedule': crontab(minute=1, hour='*'), # @hourly
+ 'kwargs': {
+ 'strategy_name': 'top_n_dashboards',
+ 'top_n': 5,
+ 'since': '7 days ago',
+ },
+ },
+ }
+
+ """
+
+ name = 'top_n_dashboards'
+
+ def __init__(self, top_n=5, since='7 days ago'):
+ super(TopNDashboardsStrategy, self).__init__()
+ self.top_n = top_n
+ self.since = parse_human_datetime(since)
+
+ def get_urls(self):
+ urls = []
+ session = db.create_scoped_session()
+
+ records = (
+ session
+ .query(Log.dashboard_id, func.count(Log.dashboard_id))
+ .filter(and_(
+ Log.dashboard_id.isnot(None),
+ Log.dttm >= self.since,
+ ))
+ .group_by(Log.dashboard_id)
+ .order_by(func.count(Log.dashboard_id).desc())
+ .limit(self.top_n)
+ .all()
+ )
+ dash_ids = [record.dashboard_id for record in records]
+ dashboards = (
+ session
+ .query(Dashboard)
+ .filter(Dashboard.id.in_(dash_ids))
+ .all()
+ )
+ for dashboard in dashboards:
+ for chart in dashboard.slices:
+ urls.append(
+ get_url({'form_data': get_form_data(chart.id, dashboard)}))
+
+ return urls
+
+
+class DashboardTagsStrategy(Strategy):
+ """
+ Warm up charts in dashboards with custom tags.
+
+ CELERYBEAT_SCHEDULE = {
+ 'cache-warmup-hourly': {
+ 'task': 'cache-warmup',
+ 'schedule': crontab(minute=1, hour='*'), # @hourly
+ 'kwargs': {
+ 'strategy_name': 'dashboard_tags',
+ 'tags': ['core', 'warmup'],
+ },
+ },
+ }
+ """
+
+ name = 'dashboard_tags'
+
+ def __init__(self, tags=None):
+ super(DashboardTagsStrategy, self).__init__()
+ self.tags = tags or []
+
+ def get_urls(self):
+ urls = []
+ session = db.create_scoped_session()
+
+ tags = (
+ session
+ .query(Tag)
+ .filter(Tag.name.in_(self.tags))
+ .all()
+ )
+ tag_ids = [tag.id for tag in tags]
+
+ # add dashboards that are tagged
+ tagged_objects = (
+ session
+ .query(TaggedObject)
+ .filter(and_(
+ TaggedObject.object_type == 'dashboard',
+ TaggedObject.tag_id.in_(tag_ids),
+ ))
+ .all()
+ )
+ dash_ids = [tagged_object.object_id for tagged_object in
tagged_objects]
+ tagged_dashboards = (
+ session
+ .query(Dashboard)
+ .filter(Dashboard.id.in_(dash_ids))
+ )
+ for dashboard in tagged_dashboards:
+ for chart in dashboard.slices:
+ urls.append(
+ get_url({'form_data': get_form_data(chart.id, dashboard)}))
+
+ # add charts that are tagged
+ tagged_objects = (
+ session
+ .query(TaggedObject)
+ .filter(and_(
+ TaggedObject.object_type == 'chart',
+ TaggedObject.tag_id.in_(tag_ids),
+ ))
+ .all()
+ )
+ chart_ids = [tagged_object.object_id for tagged_object in
tagged_objects]
+ tagged_charts = (
+ session
+ .query(Slice)
+ .filter(Slice.id.in_(chart_ids))
+ )
+ for chart in tagged_charts:
+ urls.append(get_url({'form_data': get_form_data(chart.id)}))
+
+ return urls
+
+
+strategies = [DummyStrategy, TopNDashboardsStrategy, DashboardTagsStrategy]
+
+
+@celery_app.task(name='cache-warmup')
+def cache_warmup(strategy_name, *args, **kwargs):
+ """
+ Warm up cache.
+
+ This task periodically hits charts to warm up the cache.
+
+ """
+ logger.info('Loading strategy')
+ class_ = None
+ for class_ in strategies:
+ if class_.name == strategy_name:
+ break
+ else:
+ message = f'No strategy {strategy_name} found!'
+ logger.error(message)
+ return message
+
+ logger.info(f'Loading {class_.__name__}')
+ try:
+ strategy = class_(*args, **kwargs)
+ logger.info('Success!')
+ except TypeError:
+ message = 'Error loading strategy!'
+ logger.exception(message)
+ return message
+
+ results = {'success': [], 'errors': []}
+ for url in strategy.get_urls():
+ try:
+ logger.info(f'Fetching {url}')
+ requests.get(url)
+ results['success'].append(url)
+ except RequestException:
+ logger.exception('Error warming up cache!')
+ results['errors'].append(url)
+
+ return results
diff --git a/tests/strategy_tests.py b/tests/strategy_tests.py
new file mode 100644
index 0000000..204f954
--- /dev/null
+++ b/tests/strategy_tests.py
@@ -0,0 +1,236 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Unit tests for Superset cache warmup"""
+import json
+from unittest.mock import MagicMock
+
+from superset import db
+from superset.models.core import Log
+from superset.models.tags import (
+ get_tag,
+ ObjectTypes,
+ TaggedObject,
+ TagTypes,
+)
+from superset.tasks.cache import (
+ DashboardTagsStrategy,
+ DummyStrategy,
+ get_form_data,
+ TopNDashboardsStrategy,
+)
+from .base_tests import SupersetTestCase
+
+
+TEST_URL = 'http://0.0.0.0:8081/superset/explore_json'
+
+
+class CacheWarmUpTests(SupersetTestCase):
+
+ def __init__(self, *args, **kwargs):
+ super(CacheWarmUpTests, self).__init__(*args, **kwargs)
+
+ def test_get_form_data_chart_only(self):
+ chart_id = 1
+ result = get_form_data(chart_id, None)
+ expected = {'slice_id': chart_id}
+ self.assertEqual(result, expected)
+
+ def test_get_form_data_no_dashboard_metadata(self):
+ chart_id = 1
+ dashboard = MagicMock()
+ dashboard.json_metadata = None
+ result = get_form_data(chart_id, dashboard)
+ expected = {'slice_id': chart_id}
+ self.assertEqual(result, expected)
+
+ def test_get_form_data_immune_slice(self):
+ chart_id = 1
+ filter_box_id = 2
+ dashboard = MagicMock()
+ dashboard.json_metadata = json.dumps({
+ 'filter_immune_slices': [chart_id],
+ 'default_filters': json.dumps({
+ str(filter_box_id): {'name': ['Alice', 'Bob']},
+ }),
+ })
+ result = get_form_data(chart_id, dashboard)
+ expected = {'slice_id': chart_id}
+ self.assertEqual(result, expected)
+
+ def test_get_form_data_no_default_filters(self):
+ chart_id = 1
+ dashboard = MagicMock()
+ dashboard.json_metadata = json.dumps({})
+ result = get_form_data(chart_id, dashboard)
+ expected = {'slice_id': chart_id}
+ self.assertEqual(result, expected)
+
+ def test_get_form_data_immune_fields(self):
+ chart_id = 1
+ filter_box_id = 2
+ dashboard = MagicMock()
+ dashboard.json_metadata = json.dumps({
+ 'default_filters': json.dumps({
+ str(filter_box_id): {
+ 'name': ['Alice', 'Bob'],
+ '__time_range': '100 years ago : today',
+ },
+ }),
+ 'filter_immune_slice_fields': {chart_id: ['__time_range']},
+ })
+ result = get_form_data(chart_id, dashboard)
+ expected = {
+ 'slice_id': chart_id,
+ 'extra_filters': [
+ {
+ 'col': 'name',
+ 'op': 'in',
+ 'val': ['Alice', 'Bob'],
+ },
+ ],
+ }
+ self.assertEqual(result, expected)
+
+ def test_get_form_data_no_extra_filters(self):
+ chart_id = 1
+ filter_box_id = 2
+ dashboard = MagicMock()
+ dashboard.json_metadata = json.dumps({
+ 'default_filters': json.dumps({
+ str(filter_box_id): {
+ '__time_range': '100 years ago : today',
+ },
+ }),
+ 'filter_immune_slice_fields': {chart_id: ['__time_range']},
+ })
+ result = get_form_data(chart_id, dashboard)
+ expected = {'slice_id': chart_id}
+ self.assertEqual(result, expected)
+
+ def test_get_form_data(self):
+ chart_id = 1
+ filter_box_id = 2
+ dashboard = MagicMock()
+ dashboard.json_metadata = json.dumps({
+ 'default_filters': json.dumps({
+ str(filter_box_id): {
+ 'name': ['Alice', 'Bob'],
+ '__time_range': '100 years ago : today',
+ },
+ }),
+ })
+ result = get_form_data(chart_id, dashboard)
+ expected = {
+ 'slice_id': chart_id,
+ 'extra_filters': [
+ {
+ 'col': 'name',
+ 'op': 'in',
+ 'val': ['Alice', 'Bob'],
+ },
+ {
+ 'col': '__time_range',
+ 'op': 'in',
+ 'val': '100 years ago : today',
+ },
+ ],
+ }
+ self.assertEqual(result, expected)
+
+ def test_dummy_strategy(self):
+ strategy = DummyStrategy()
+ result = sorted(strategy.get_urls())
+ expected = [
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+1%7D',
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+17%7D',
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+18%7D',
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+19%7D',
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+30%7D',
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+31%7D',
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+8%7D',
+ ]
+ self.assertEqual(result, expected)
+
+ def test_top_n_dashboards_strategy(self):
+ # create a top visited dashboard
+ db.session.query(Log).delete()
+ self.login(username='admin')
+ for _ in range(10):
+ self.client.get('/superset/dashboard/3/')
+
+ strategy = TopNDashboardsStrategy(1)
+ result = sorted(strategy.get_urls())
+ expected = [
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+31%7D',
+ ]
+ self.assertEqual(result, expected)
+
+ def test_dashboard_tags(self):
+ strategy = DashboardTagsStrategy(['tag1'])
+
+ result = sorted(strategy.get_urls())
+ expected = []
+ self.assertEqual(result, expected)
+
+ # tag dashboard 3 with `tag1`
+ tag1 = get_tag('tag1', db.session, TagTypes.custom)
+ object_id = 3
+ tagged_object = TaggedObject(
+ tag_id=tag1.id,
+ object_id=object_id,
+ object_type=ObjectTypes.dashboard,
+ )
+ db.session.add(tagged_object)
+ db.session.commit()
+
+ result = sorted(strategy.get_urls())
+ expected = [
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+31%7D',
+ ]
+ self.assertEqual(result, expected)
+
+ strategy = DashboardTagsStrategy(['tag2'])
+
+ result = sorted(strategy.get_urls())
+ expected = []
+ self.assertEqual(result, expected)
+
+ # tag chart 30 with `tag2`
+ tag2 = get_tag('tag2', db.session, TagTypes.custom)
+ object_id = 30
+ tagged_object = TaggedObject(
+ tag_id=tag2.id,
+ object_id=object_id,
+ object_type=ObjectTypes.chart,
+ )
+ db.session.add(tagged_object)
+ db.session.commit()
+
+ result = sorted(strategy.get_urls())
+ expected = [
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+30%7D',
+ ]
+ self.assertEqual(result, expected)
+
+ strategy = DashboardTagsStrategy(['tag1', 'tag2'])
+
+ result = sorted(strategy.get_urls())
+ expected = [
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+30%7D',
+ f'{TEST_URL}/?form_data=%7B%27slice_id%27%3A+31%7D',
+ ]
+ self.assertEqual(result, expected)
