This is an automated email from the ASF dual-hosted git repository. rusackas pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-superset-site.git
commit 23eb7fb7f4a16a65352c989e472682ac23862b3a Author: Ville Brofeldt <[email protected]> AuthorDate: Sun Oct 13 17:37:46 2019 +0300 0.34.1 doc version --- .buildinfo | 2 +- _modules/superset/jinja_context.html | 78 ++++++++++++++++++++++++--- _sources/installation.rst.txt | 90 ++++++++++++++++++++++++++++++- _sources/sqllab.rst.txt | 4 +- genindex.html | 21 +++++++- index.html | 3 ++ installation.html | 86 +++++++++++++++++++++++++++++- objects.inv | Bin 583 -> 634 bytes searchindex.js | 2 +- sqllab.html | 100 +++++++++++++++++++++++++++++++++-- 10 files changed, 369 insertions(+), 17 deletions(-) diff --git a/.buildinfo b/.buildinfo index c0f5130..79eea93 100644 --- a/.buildinfo +++ b/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: aef1dd98ccac6c00170d15d9ca65cf37 +config: 57f4258c8ecbc6bcd73251593f94511a tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/_modules/superset/jinja_context.html b/_modules/superset/jinja_context.html index 8d57aa9..ec78ef9 100644 --- a/_modules/superset/jinja_context.html +++ b/_modules/superset/jinja_context.html @@ -281,7 +281,43 @@ <span class="k">return</span> <span class="p">[]</span></div> -<span class="k">class</span> <span class="nc">BaseTemplateProcessor</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span> +<span class="k">class</span> <span class="nc">CacheKeyWrapper</span><span class="p">:</span> + <span class="sd">""" Dummy class that exposes a method used to store additional values used in</span> +<span class="sd"> calculation of query object cache keys"""</span> + + <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">extra_cache_keys</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">):</span> + <span class="bp">self</span><span class="o">.</span><span class="n">extra_cache_keys</span> <span class="o">=</span> <span class="n">extra_cache_keys</span> + + <span class="k">def</span> <span class="nf">cache_key_wrapper</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">:</span> <span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">Any</span><span class="p">:</span> + <span class="sd">""" Adds values to a list that is added to the query object used for calculating</span> +<span class="sd"> a cache key.</span> + +<span class="sd"> This is needed if the following applies:</span> +<span class="sd"> - Caching is enabled</span> +<span class="sd"> - The query is dynamically generated using a jinja template</span> +<span class="sd"> - A username or similar is used as a filter in the query</span> + +<span class="sd"> Example when using a SQL query as a data source ::</span> + +<span class="sd"> SELECT action, count(*) as times</span> +<span class="sd"> FROM logs</span> +<span class="sd"> WHERE logged_in_user = '{{ cache_key_wrapper(current_username()) }}'</span> +<span class="sd"> GROUP BY action</span> + +<span class="sd"> This will ensure that the query results that were cached by `user_1` will</span> +<span class="sd"> **not** be seen by `user_2`, as the `cache_key` for the query will be</span> +<span class="sd"> different. ``cache_key_wrapper`` can be used similarly for regular table data</span> +<span class="sd"> sources by adding a `Custom SQL` filter.</span> + +<span class="sd"> :param key: Any value that should be considered when calculating the cache key</span> +<span class="sd"> :return: the original value ``key`` passed to the function</span> +<span class="sd"> """</span> + <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">extra_cache_keys</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span> + <span class="bp">self</span><span class="o">.</span><span class="n">extra_cache_keys</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> + <span class="k">return</span> <span class="n">key</span> + + +<span class="k">class</span> <span class="nc">BaseTemplateProcessor</span><span class="p">:</span> <span class="sd">"""Base class for database-specific jinja context</span> <span class="sd"> There's this bit of magic in ``process_template`` that instantiates only</span> @@ -298,7 +334,14 @@ <span class="n">engine</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span> - <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">database</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">query</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="o">**</span><span cla [...] + <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span> + <span class="bp">self</span><span class="p">,</span> + <span class="n">database</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> + <span class="n">query</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> + <span class="n">table</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> + <span class="n">extra_cache_keys</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> + <span class="o">**</span><span class="n">kwargs</span> + <span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">database</span> <span class="o">=</span> <span class="n">database</span> <span class="bp">self</span><span class="o">.</span><span class="n">query</span> <span class="o">=</span> <span class="n">query</span> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span> <span class="o">=</span> <span class="kc">None</span> @@ -310,6 +353,7 @@ <span class="s2">"url_param"</span><span class="p">:</span> <span class="n">url_param</span><span class="p">,</span> <span class="s2">"current_user_id"</span><span class="p">:</span> <span class="n">current_user_id</span><span class="p">,</span> <span class="s2">"current_username"</span><span class="p">:</span> <span class="n">current_username</span><span class="p">,</span> + <span class="s2">"cache_key_wrapper"</span><span class="p">:</span> <span class="n">CacheKeyWrapper</span><span class="p">(</span><span class="n">extra_cache_keys</span><span class="p">)</span><span class="o">.</span><span class="n">cache_key_wrapper</span><span class="p">,</span> <span class="s2">"filter_values"</span><span class="p">:</span> <span class="n">filter_values</span><span class="p">,</span> <span class="s2">"form_data"</span><span class="p">:</span> <span class="p">{},</span> <span class="p">}</span> @@ -341,22 +385,44 @@ <span class="n">engine</span> <span class="o">=</span> <span class="s2">"presto"</span> <span class="nd">@staticmethod</span> - <span class="k">def</span> <span class="nf">_schema_table</span><span class="p">(</span><span class="n">table_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">schema</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]:</span> + <span class="k">def</span> <span class="nf">_schema_table</span><span class="p">(</span> + <span class="n">table_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> + <span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span> <span class="k">if</span> <span class="s2">"."</span> <span class="ow">in</span> <span class="n">table_name</span><span class="p">:</span> <span class="n">schema</span><span class="p">,</span> <span class="n">table_name</span> <span class="o">=</span> <span class="n">table_name</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"."</span><span class="p">)</span> <span class="k">return</span> <span class="n">table_name</span><span class="p">,</span> <span class="n">schema</span> - <span class="k">def</span> <span class="nf">latest_partition</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span> +<div class="viewcode-block" id="PrestoTemplateProcessor.first_latest_partition"><a class="viewcode-back" href="../../sqllab.html#superset.jinja_context.PrestoTemplateProcessor.first_latest_partition">[docs]</a> <span class="k">def</span> <span class="nf">first_latest_partition</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-& [...] + <span class="sd">"""</span> +<span class="sd"> Gets the first value in the array of all latest partitions</span> + +<span class="sd"> :param table_name: table name in the format `schema.table`</span> +<span class="sd"> :return: the first (or only) value in the latest partition array</span> +<span class="sd"> :raises IndexError: If no partition exists</span> +<span class="sd"> """</span> + + <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">latest_partitions</span><span class="p">(</span><span class="n">table_name</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span></div> + +<div class="viewcode-block" id="PrestoTemplateProcessor.latest_partitions"><a class="viewcode-back" href="../../sqllab.html#superset.jinja_context.PrestoTemplateProcessor.latest_partitions">[docs]</a> <span class="k">def</span> <span class="nf">latest_partitions</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <spa [...] + <span class="sd">"""</span> +<span class="sd"> Gets the array of all latest partitions</span> + +<span class="sd"> :param table_name: table name in the format `schema.table`</span> +<span class="sd"> :return: the latest partition array</span> +<span class="sd"> """</span> + <span class="n">table_name</span><span class="p">,</span> <span class="n">schema</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_schema_table</span><span class="p">(</span><span class="n">table_name</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span><span class="p">)</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">database</span><span class="o">.</span><span class="n">db_engine_spec</span><span class="o">.</span><span class="n">latest_partition</span><span class="p">(</span> <span class="n">table_name</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">database</span> - <span class="p">)[</span><span class="mi">1</span><span class="p">]</span> + <span class="p">)[</span><span class="mi">1</span><span class="p">]</span></div> <span class="k">def</span> <span class="nf">latest_sub_partition</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table_name</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span> <span class="n">table_name</span><span class="p">,</span> <span class="n">schema</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_schema_table</span><span class="p">(</span><span class="n">table_name</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span><span class="p">)</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">database</span><span class="o">.</span><span class="n">db_engine_spec</span><span class="o">.</span><span class="n">latest_sub_partition</span><span class="p">(</span> <span class="n">table_name</span><span class="o">=</span><span class="n">table_name</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">,</span> <span class="n">database</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">database</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span> - <span class="p">)</span></div> + <span class="p">)</span> + + <span class="n">latest_partition</span> <span class="o">=</span> <span class="n">first_latest_partition</span></div> <div class="viewcode-block" id="HiveTemplateProcessor"><a class="viewcode-back" href="../../sqllab.html#superset.jinja_context.HiveTemplateProcessor">[docs]</a><span class="k">class</span> <span class="nc">HiveTemplateProcessor</span><span class="p">(</span><span class="n">PrestoTemplateProcessor</span><span class="p">):</span> diff --git a/_sources/installation.rst.txt b/_sources/installation.rst.txt index 37991de..e560657 100644 --- a/_sources/installation.rst.txt +++ b/_sources/installation.rst.txt @@ -521,6 +521,24 @@ into your global default defined in ``CACHE_CONFIG``. 'CACHE_REDIS_URL': 'redis://localhost:6379/0', } +It is also possible to pass a custom cache initialization function in the +config to handle additional caching use cases. The function must return an +object that is compatible with the `Flask-Cache <https://pythonhosted.org/Flask-Cache/>`_ API. + +.. code-block:: python + + from custom_caching import CustomCache + + def init_cache(app): + """Takes an app instance and returns a custom cache backend""" + config = { + 'CACHE_DEFAULT_TIMEOUT': 60 * 60 * 24, # 1 day default (in secs) + 'CACHE_KEY_PREFIX': 'superset_results', + } + return CustomCache(app, config) + + CACHE_CONFIG = init_cache + Superset has a Celery task that will periodically warm up the cache based on different strategies. To use it, add the following to the `CELERYBEAT_SCHEDULE` section in `config.py`: @@ -629,6 +647,18 @@ Note that you can run the ``superset refresh_druid`` command to refresh the metadata from your Druid cluster(s) +Presto +------ + +By default Superset assumes the most recent version of Presto is being used when +querying the datasource. If you're using an older version of presto, you can configure +it in the ``extra`` parameter:: + + { + "version": "0.123" + } + + CORS ---- @@ -649,7 +679,7 @@ DOMAIN SHARDING Chrome allows up to 6 open connections per domain at a time. When there are more than 6 slices in dashboard, a lot of time fetch requests are queued up and wait for -next available socket. PR (`#5039 <https://github.com/apache/incubator-superset/pull/5039>`) adds domain sharding to Superset, +next available socket. `PR 5039 <https://github.com/apache/incubator-superset/pull/5039>`_ adds domain sharding to Superset, and this feature will be enabled by configuration only (by default Superset doesn't allow cross-domain request). @@ -678,6 +708,40 @@ environment variable: :: *Adapted from http://flask.pocoo.org/snippets/69/* +Event Logging +------------- + +Superset by default logs special action event on it's database. These log can be accessed on the UI navigating to +"Security" -> "Action Log". You can freely customize these logs by implementing your own event log class. + +Example of a simple JSON to Stdout class:: + + class JSONStdOutEventLogger(AbstractEventLogger): + + def log(self, user_id, action, *args, **kwargs): + records = kwargs.get('records', list()) + dashboard_id = kwargs.get('dashboard_id') + slice_id = kwargs.get('slice_id') + duration_ms = kwargs.get('duration_ms') + referrer = kwargs.get('referrer') + + for record in records: + log = dict( + action=action, + json=record, + dashboard_id=dashboard_id, + slice_id=slice_id, + duration_ms=duration_ms, + referrer=referrer, + user_id=user_id + ) + print(json.dumps(log)) + + +Then on Superset's config pass an instance of the logger type you want to use. + + EVENT_LOGGER = JSONStdOutEventLogger() + Upgrading --------- @@ -990,7 +1054,7 @@ your environment. See `CONTRIBUTING.md#setup-local-environment-for-development < Blueprints ---------- -`Blueprints are Flask's reusable apps <http://flask.pocoo.org/docs/0.12/blueprints/>`_. +`Blueprints are Flask's reusable apps <https://flask.palletsprojects.com/en/1.0.x/tutorial/views/>`_. Superset allows you to specify an array of Blueprints in your ``superset_config`` module. Here's an example of how this can work with a simple Blueprint. By doing @@ -1109,3 +1173,25 @@ Then we can add this two lines to ``superset_config.py``: from custom_sso_security_manager import CustomSsoSecurityManager CUSTOM_SECURITY_MANAGER = CustomSsoSecurityManager + +Feature Flags +--------------------------- + +Because of a wide variety of users, Superset has some features that are not enabled by default. For example, some users have stronger security restrictions, while some others may not. So Superset allow users to enable or disable some features by config. For feature owners, you can add optional functionalities in Superset, but will be only affected by a subset of users. + +You can enable or disable features with flag from ``superset_config.py``: + +.. code-block:: python + + DEFAULT_FEATURE_FLAGS = { + 'CLIENT_CACHE': False, + 'ENABLE_EXPLORE_JSON_CSRF_PROTECTION': False + } + +Here is a list of flags and descriptions: + +* ENABLE_EXPLORE_JSON_CSRF_PROTECTION + + * For some security concerns, you may need to enforce CSRF protection on all query request to explore_json endpoint. In Superset, we use `flask-csrf <https://sjl.bitbucket.io/flask-csrf/>`_ add csrf protection for all POST requests, but this protection doesn't apply to GET method. + + * When ENABLE_EXPLORE_JSON_CSRF_PROTECTION is set to true, your users cannot make GET request to explore_json. The default value for this feature False (current behavior), explore_json accepts both GET and POST request. See `PR 7935 <https://github.com/apache/incubator-superset/pull/7935>`_ for more details. diff --git a/_sources/sqllab.rst.txt b/_sources/sqllab.rst.txt index 5fe24ad..c69e584 100644 --- a/_sources/sqllab.rst.txt +++ b/_sources/sqllab.rst.txt @@ -56,7 +56,7 @@ Templating with Jinja SELECT * FROM some_table - WHERE partition_key = '{{ presto.latest_partition('some_table') }}' + WHERE partition_key = '{{ presto.first_latest_partition('some_table') }}' Templating unleashes the power and capabilities of a programming language within your SQL code. @@ -87,6 +87,8 @@ Superset's Jinja context: .. autofunction:: superset.jinja_context.filter_values +.. autofunction:: superset.jinja_context.CacheKeyWrapper.cache_key_wrapper + .. autoclass:: superset.jinja_context.PrestoTemplateProcessor :members: diff --git a/genindex.html b/genindex.html index 6c030ff..bf9efde 100644 --- a/genindex.html +++ b/genindex.html @@ -157,6 +157,7 @@ <a href="#C"><strong>C</strong></a> | <a href="#F"><strong>F</strong></a> | <a href="#H"><strong>H</strong></a> + | <a href="#L"><strong>L</strong></a> | <a href="#P"><strong>P</strong></a> | <a href="#U"><strong>U</strong></a> @@ -164,10 +165,12 @@ <h2 id="C">C</h2> <table style="width: 100%" class="indextable genindextable"><tr> <td style="width: 33%; vertical-align: top;"><ul> - <li><a href="sqllab.html#superset.jinja_context.current_user_id">current_user_id() (in module superset.jinja_context)</a> + <li><a href="sqllab.html#superset.jinja_context.CacheKeyWrapper.cache_key_wrapper">cache_key_wrapper() (in module superset.jinja_context.CacheKeyWrapper)</a> </li> </ul></td> <td style="width: 33%; vertical-align: top;"><ul> + <li><a href="sqllab.html#superset.jinja_context.current_user_id">current_user_id() (in module superset.jinja_context)</a> +</li> <li><a href="sqllab.html#superset.jinja_context.current_username">current_username() (in module superset.jinja_context)</a> </li> </ul></td> @@ -179,6 +182,10 @@ <li><a href="sqllab.html#superset.jinja_context.filter_values">filter_values() (in module superset.jinja_context)</a> </li> </ul></td> + <td style="width: 33%; vertical-align: top;"><ul> + <li><a href="sqllab.html#superset.jinja_context.PrestoTemplateProcessor.first_latest_partition">first_latest_partition() (superset.jinja_context.PrestoTemplateProcessor method)</a> +</li> + </ul></td> </tr></table> <h2 id="H">H</h2> @@ -189,6 +196,18 @@ </ul></td> </tr></table> +<h2 id="L">L</h2> +<table style="width: 100%" class="indextable genindextable"><tr> + <td style="width: 33%; vertical-align: top;"><ul> + <li><a href="sqllab.html#superset.jinja_context.PrestoTemplateProcessor.latest_partition">latest_partition() (superset.jinja_context.PrestoTemplateProcessor method)</a> +</li> + </ul></td> + <td style="width: 33%; vertical-align: top;"><ul> + <li><a href="sqllab.html#superset.jinja_context.PrestoTemplateProcessor.latest_partitions">latest_partitions() (superset.jinja_context.PrestoTemplateProcessor method)</a> +</li> + </ul></td> +</tr></table> + <h2 id="P">P</h2> <table style="width: 100%" class="indextable genindextable"><tr> <td style="width: 33%; vertical-align: top;"><ul> diff --git a/index.html b/index.html index 485b8b9..1586fae 100644 --- a/index.html +++ b/index.html @@ -291,9 +291,11 @@ be supported as well.</p> <li class="toctree-l2"><a class="reference internal" href="installation.html#external-password-store-for-sqlalchemy-connections">External Password store for SQLAlchemy connections</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#ssl-access-to-databases">SSL Access to databases</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#druid">Druid</a></li> +<li class="toctree-l2"><a class="reference internal" href="installation.html#presto">Presto</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#cors">CORS</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#domain-sharding">DOMAIN SHARDING</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#middleware">MIDDLEWARE</a></li> +<li class="toctree-l2"><a class="reference internal" href="installation.html#event-logging">Event Logging</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#upgrading">Upgrading</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#celery-tasks">Celery Tasks</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#email-reports">Email Reports</a></li> @@ -304,6 +306,7 @@ be supported as well.</p> <li class="toctree-l2"><a class="reference internal" href="installation.html#statsd-logging">StatsD logging</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#install-superset-with-helm-in-kubernetes">Install Superset with helm in Kubernetes</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#custom-oauth2-configuration">Custom OAuth2 configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="installation.html#feature-flags">Feature Flags</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial - Creating your first dashboard</a><ul> diff --git a/installation.html b/installation.html index a9967bc..2b17546 100644 --- a/installation.html +++ b/installation.html @@ -105,9 +105,11 @@ <li class="toctree-l2"><a class="reference internal" href="#external-password-store-for-sqlalchemy-connections">External Password store for SQLAlchemy connections</a></li> <li class="toctree-l2"><a class="reference internal" href="#ssl-access-to-databases">SSL Access to databases</a></li> <li class="toctree-l2"><a class="reference internal" href="#druid">Druid</a></li> +<li class="toctree-l2"><a class="reference internal" href="#presto">Presto</a></li> <li class="toctree-l2"><a class="reference internal" href="#cors">CORS</a></li> <li class="toctree-l2"><a class="reference internal" href="#domain-sharding">DOMAIN SHARDING</a></li> <li class="toctree-l2"><a class="reference internal" href="#middleware">MIDDLEWARE</a></li> +<li class="toctree-l2"><a class="reference internal" href="#event-logging">Event Logging</a></li> <li class="toctree-l2"><a class="reference internal" href="#upgrading">Upgrading</a></li> <li class="toctree-l2"><a class="reference internal" href="#celery-tasks">Celery Tasks</a></li> <li class="toctree-l2"><a class="reference internal" href="#email-reports">Email Reports</a></li> @@ -118,6 +120,7 @@ <li class="toctree-l2"><a class="reference internal" href="#statsd-logging">StatsD logging</a></li> <li class="toctree-l2"><a class="reference internal" href="#install-superset-with-helm-in-kubernetes">Install Superset with helm in Kubernetes</a></li> <li class="toctree-l2"><a class="reference internal" href="#custom-oauth2-configuration">Custom OAuth2 configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="#feature-flags">Feature Flags</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial - Creating your first dashboard</a></li> @@ -710,6 +713,22 @@ into your global default defined in <code class="docutils literal notranslate">< <span class="p">}</span> </pre></div> </div> +<p>It is also possible to pass a custom cache initialization function in the +config to handle additional caching use cases. The function must return an +object that is compatible with the <a class="reference external" href="https://pythonhosted.org/Flask-Cache/";>Flask-Cache</a> API.</p> +<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">custom_caching</span> <span class="kn">import</span> <span class="n">CustomCache</span> + +<span class="k">def</span> <span class="nf">init_cache</span><span class="p">(</span><span class="n">app</span><span class="p">):</span> + <span class="sd">"""Takes an app instance and returns a custom cache backend"""</span> + <span class="n">config</span> <span class="o">=</span> <span class="p">{</span> + <span class="s1">'CACHE_DEFAULT_TIMEOUT'</span><span class="p">:</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">60</span> <span class="o">*</span> <span class="mi">24</span><span class="p">,</span> <span class="c1"># 1 day default (in secs)</span> + <span class="s1">'CACHE_KEY_PREFIX'</span><span class="p">:</span> <span class="s1">'superset_results'</span><span class="p">,</span> + <span class="p">}</span> + <span class="k">return</span> <span class="n">CustomCache</span><span class="p">(</span><span class="n">app</span><span class="p">,</span> <span class="n">config</span><span class="p">)</span> + +<span class="n">CACHE_CONFIG</span> <span class="o">=</span> <span class="n">init_cache</span> +</pre></div> +</div> <p>Superset has a Celery task that will periodically warm up the cache based on different strategies. To use it, add the following to the <cite>CELERYBEAT_SCHEDULE</cite> section in <cite>config.py</cite>:</p> @@ -804,6 +823,17 @@ parameter</p> <p>Note that you can run the <code class="docutils literal notranslate"><span class="pre">superset</span> <span class="pre">refresh_druid</span></code> command to refresh the metadata from your Druid cluster(s)</p> </div> +<div class="section" id="presto"> +<h2>Presto<a class="headerlink" href="#presto" title="Permalink to this headline">¶</a></h2> +<p>By default Superset assumes the most recent version of Presto is being used when +querying the datasource. If you’re using an older version of presto, you can configure +it in the <code class="docutils literal notranslate"><span class="pre">extra</span></code> parameter:</p> +<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span> + <span class="s2">"version"</span><span class="p">:</span> <span class="s2">"0.123"</span> +<span class="p">}</span> +</pre></div> +</div> +</div> <div class="section" id="cors"> <h2>CORS<a class="headerlink" href="#cors" title="Permalink to this headline">¶</a></h2> <p>The extra CORS Dependency must be installed:</p> @@ -820,7 +850,7 @@ metadata from your Druid cluster(s)</p> <h2>DOMAIN SHARDING<a class="headerlink" href="#domain-sharding" title="Permalink to this headline">¶</a></h2> <p>Chrome allows up to 6 open connections per domain at a time. When there are more than 6 slices in dashboard, a lot of time fetch requests are queued up and wait for -next available socket. PR (<cite>#5039 <https://github.com/apache/incubator-superset/pull/5039></cite>) adds domain sharding to Superset, +next available socket. <a class="reference external" href="https://github.com/apache/incubator-superset/pull/5039";>PR 5039</a> adds domain sharding to Superset, and this feature will be enabled by configuration only (by default Superset doesn’t allow cross-domain request).</p> <ul class="simple"> @@ -847,6 +877,38 @@ environment variable:</p> </div> <p><em>Adapted from http://flask.pocoo.org/snippets/69/</em></p> </div> +<div class="section" id="event-logging"> +<h2>Event Logging<a class="headerlink" href="#event-logging" title="Permalink to this headline">¶</a></h2> +<p>Superset by default logs special action event on it’s database. These log can be accessed on the UI navigating to +“Security” -> “Action Log”. You can freely customize these logs by implementing your own event log class.</p> +<p>Example of a simple JSON to Stdout class:</p> +<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">JSONStdOutEventLogger</span><span class="p">(</span><span class="n">AbstractEventLogger</span><span class="p">):</span> + + <span class="k">def</span> <span class="nf">log</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">user_id</span><span class="p">,</span> <span class="n">action</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span> + <span class="n">records</span> <span class="o">=</span> <span class="n">kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'records'</span><span class="p">,</span> <span class="nb">list</span><span class="p">())</span> + <span class="n">dashboard_id</span> <span class="o">=</span> <span class="n">kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'dashboard_id'</span><span class="p">)</span> + <span class="n">slice_id</span> <span class="o">=</span> <span class="n">kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'slice_id'</span><span class="p">)</span> + <span class="n">duration_ms</span> <span class="o">=</span> <span class="n">kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'duration_ms'</span><span class="p">)</span> + <span class="n">referrer</span> <span class="o">=</span> <span class="n">kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'referrer'</span><span class="p">)</span> + + <span class="k">for</span> <span class="n">record</span> <span class="ow">in</span> <span class="n">records</span><span class="p">:</span> + <span class="n">log</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span> + <span class="n">action</span><span class="o">=</span><span class="n">action</span><span class="p">,</span> + <span class="n">json</span><span class="o">=</span><span class="n">record</span><span class="p">,</span> + <span class="n">dashboard_id</span><span class="o">=</span><span class="n">dashboard_id</span><span class="p">,</span> + <span class="n">slice_id</span><span class="o">=</span><span class="n">slice_id</span><span class="p">,</span> + <span class="n">duration_ms</span><span class="o">=</span><span class="n">duration_ms</span><span class="p">,</span> + <span class="n">referrer</span><span class="o">=</span><span class="n">referrer</span><span class="p">,</span> + <span class="n">user_id</span><span class="o">=</span><span class="n">user_id</span> + <span class="p">)</span> + <span class="nb">print</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">log</span><span class="p">))</span> +</pre></div> +</div> +<p>Then on Superset’s config pass an instance of the logger type you want to use.</p> +<blockquote> +<div><p>EVENT_LOGGER = JSONStdOutEventLogger()</p> +</div></blockquote> +</div> <div class="section" id="upgrading"> <h2>Upgrading<a class="headerlink" href="#upgrading" title="Permalink to this headline">¶</a></h2> <p>Upgrading should be as straightforward as running:</p> @@ -1147,7 +1209,7 @@ your environment. See <a class="reference external" href="https://github.com/apa </div> <div class="section" id="blueprints"> <h2>Blueprints<a class="headerlink" href="#blueprints" title="Permalink to this headline">¶</a></h2> -<p><a class="reference external" href="http://flask.pocoo.org/docs/0.12/blueprints/";>Blueprints are Flask’s reusable apps</a>. +<p><a class="reference external" href="https://flask.palletsprojects.com/en/1.0.x/tutorial/views/";>Blueprints are Flask’s reusable apps</a>. Superset allows you to specify an array of Blueprints in your <code class="docutils literal notranslate"><span class="pre">superset_config</span></code> module. Here’s an example of how this can work with a simple Blueprint. By doing @@ -1249,6 +1311,26 @@ located in <code class="docutils literal notranslate"><span class="pre">install/ </pre></div> </div> </div> +<div class="section" id="feature-flags"> +<h2>Feature Flags<a class="headerlink" href="#feature-flags" title="Permalink to this headline">¶</a></h2> +<p>Because of a wide variety of users, Superset has some features that are not enabled by default. For example, some users have stronger security restrictions, while some others may not. So Superset allow users to enable or disable some features by config. For feature owners, you can add optional functionalities in Superset, but will be only affected by a subset of users.</p> +<p>You can enable or disable features with flag from <code class="docutils literal notranslate"><span class="pre">superset_config.py</span></code>:</p> +<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">DEFAULT_FEATURE_FLAGS</span> <span class="o">=</span> <span class="p">{</span> + <span class="s1">'CLIENT_CACHE'</span><span class="p">:</span> <span class="bp">False</span><span class="p">,</span> + <span class="s1">'ENABLE_EXPLORE_JSON_CSRF_PROTECTION'</span><span class="p">:</span> <span class="bp">False</span> +<span class="p">}</span> +</pre></div> +</div> +<p>Here is a list of flags and descriptions:</p> +<ul class="simple"> +<li><p>ENABLE_EXPLORE_JSON_CSRF_PROTECTION</p> +<ul> +<li><p>For some security concerns, you may need to enforce CSRF protection on all query request to explore_json endpoint. In Superset, we use <a class="reference external" href="https://sjl.bitbucket.io/flask-csrf/";>flask-csrf</a> add csrf protection for all POST requests, but this protection doesn’t apply to GET method.</p></li> +<li><p>When ENABLE_EXPLORE_JSON_CSRF_PROTECTION is set to true, your users cannot make GET request to explore_json. The default value for this feature False (current behavior), explore_json accepts both GET and POST request. See <a class="reference external" href="https://github.com/apache/incubator-superset/pull/7935";>PR 7935</a> for more details.</p></li> +</ul> +</li> +</ul> +</div> </div> diff --git a/objects.inv b/objects.inv index 62d5b78..55df316 100644 Binary files a/objects.inv and b/objects.inv differ diff --git a/searchindex.js b/searchindex.js index 1858724..c8a2caa 100644 --- a/searchindex.js +++ b/searchindex.js @@ -1 +1 @@ -Search.setIndex({docnames:["druid","faq","gallery","import_export_datasources","index","installation","misc","security","sqllab","tutorial","videos","visualization"],envversion:{"sphinx.domains.c":1,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":1,"sphinx.domains.javascript":1,"sphinx.domains.math":2,"sphinx.domains.python":1,"sphinx.domains.rst":1,"sphinx.domains.std":1,"sphinx.ext.viewcode":1,sphinx:56},filenames:["druid.rst","faq.rst","gallery.rst","impo [...] \ No newline at end of file +Search.setIndex({docnames:["druid","faq","gallery","import_export_datasources","index","installation","misc","security","sqllab","tutorial","videos","visualization"],envversion:{"sphinx.domains.c":1,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":1,"sphinx.domains.javascript":1,"sphinx.domains.math":2,"sphinx.domains.python":1,"sphinx.domains.rst":1,"sphinx.domains.std":1,"sphinx.ext.viewcode":1,sphinx:56},filenames:["druid.rst","faq.rst","gallery.rst","impo [...] \ No newline at end of file diff --git a/sqllab.html b/sqllab.html index 41f123d..1fe9e9f 100644 --- a/sqllab.html +++ b/sqllab.html @@ -198,7 +198,7 @@ which allows for using macros in your SQL code</p></li> <h2>Templating with Jinja<a class="headerlink" href="#templating-with-jinja" title="Permalink to this headline">¶</a></h2> <div class="highlight-sql notranslate"><div class="highlight"><pre><span></span><span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">some_table</span> -<span class="k">WHERE</span> <span class="n">partition_key</span> <span class="o">=</span> <span class="s1">'{{ presto.latest_partition('</span><span class="n">some_table</span><span class="s1">') }}'</span> +<span class="k">WHERE</span> <span class="n">partition_key</span> <span class="o">=</span> <span class="s1">'{{ presto.first_latest_partition('</span><span class="n">some_table</span><span class="s1">') }}'</span> </pre></div> </div> <p>Templating unleashes the power and capabilities of a @@ -302,17 +302,111 @@ purposes</p></li> </dl> </dd></dl> +<dl class="function"> +<dt id="superset.jinja_context.CacheKeyWrapper.cache_key_wrapper"> +<code class="sig-prename descclassname">superset.jinja_context.CacheKeyWrapper.</code><code class="sig-name descname">cache_key_wrapper</code><span class="sig-paren">(</span><em class="sig-param">self</em>, <em class="sig-param">key</em><span class="sig-paren">)</span><a class="headerlink" href="#superset.jinja_context.CacheKeyWrapper.cache_key_wrapper" title="Permalink to this definition">¶</a></dt> +<dd><p>Adds values to a list that is added to the query object used for calculating +a cache key.</p> +<dl class="simple"> +<dt>This is needed if the following applies:</dt><dd><ul class="simple"> +<li><p>Caching is enabled</p></li> +<li><p>The query is dynamically generated using a jinja template</p></li> +<li><p>A username or similar is used as a filter in the query</p></li> +</ul> +</dd> +</dl> +<p>Example when using a SQL query as a data source</p> +<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SELECT</span> <span class="n">action</span><span class="p">,</span> <span class="n">count</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="k">as</span> <span class="n">times</span> +<span class="n">FROM</span> <span class="n">logs</span> +<span class="n">WHERE</span> <span class="n">logged_in_user</span> <span class="o">=</span> <span class="s1">'{{ cache_key_wrapper(current_username()) }}'</span> +<span class="n">GROUP</span> <span class="n">BY</span> <span class="n">action</span> +</pre></div> +</div> +<p>This will ensure that the query results that were cached by <cite>user_1</cite> will +<strong>not</strong> be seen by <cite>user_2</cite>, as the <cite>cache_key</cite> for the query will be +different. <code class="docutils literal notranslate"><span class="pre">cache_key_wrapper</span></code> can be used similarly for regular table data +sources by adding a <cite>Custom SQL</cite> filter.</p> +<dl class="field-list simple"> +<dt class="field-odd">Parameters</dt> +<dd class="field-odd"><p><strong>key</strong> (<code class="xref py py-data docutils literal notranslate"><span class="pre">Any</span></code>) – Any value that should be considered when calculating the cache key</p> +</dd> +<dt class="field-even">Return type</dt> +<dd class="field-even"><p><code class="xref py py-data docutils literal notranslate"><span class="pre">Any</span></code></p> +</dd> +<dt class="field-odd">Returns</dt> +<dd class="field-odd"><p>the original value <code class="docutils literal notranslate"><span class="pre">key</span></code> passed to the function</p> +</dd> +</dl> +</dd></dl> + <dl class="class"> <dt id="superset.jinja_context.PrestoTemplateProcessor"> -<em class="property">class </em><code class="sig-prename descclassname">superset.jinja_context.</code><code class="sig-name descname">PrestoTemplateProcessor</code><span class="sig-paren">(</span><em class="sig-param">database=None</em>, <em class="sig-param">query=None</em>, <em class="sig-param">table=None</em>, <em class="sig-param">**kwargs</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/superset/jinja_context.html#PrestoTemplateProcessor"><span class [...] +<em class="property">class </em><code class="sig-prename descclassname">superset.jinja_context.</code><code class="sig-name descname">PrestoTemplateProcessor</code><span class="sig-paren">(</span><em class="sig-param">database=None</em>, <em class="sig-param">query=None</em>, <em class="sig-param">table=None</em>, <em class="sig-param">extra_cache_keys=None</em>, <em class="sig-param">**kwargs</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/superset/jinja [...] <dd><p>Presto Jinja context</p> <p>The methods described here are namespaced under <code class="docutils literal notranslate"><span class="pre">presto</span></code> in the jinja context as in <code class="docutils literal notranslate"><span class="pre">SELECT</span> <span class="pre">'{{</span> <span class="pre">presto.some_macro_call()</span> <span class="pre">}}'</span></code></p> +<dl class="method"> +<dt id="superset.jinja_context.PrestoTemplateProcessor.first_latest_partition"> +<code class="sig-name descname">first_latest_partition</code><span class="sig-paren">(</span><em class="sig-param">table_name</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/superset/jinja_context.html#PrestoTemplateProcessor.first_latest_partition"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#superset.jinja_context.PrestoTemplateProcessor.first_latest_partition" title="Permalink to this definition">¶</a></dt> +<dd><p>Gets the first value in the array of all latest partitions</p> +<dl class="field-list simple"> +<dt class="field-odd">Parameters</dt> +<dd class="field-odd"><p><strong>table_name</strong> (<code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code>) – table name in the format <cite>schema.table</cite></p> +</dd> +<dt class="field-even">Return type</dt> +<dd class="field-even"><p><code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code></p> +</dd> +<dt class="field-odd">Returns</dt> +<dd class="field-odd"><p>the first (or only) value in the latest partition array</p> +</dd> +<dt class="field-even">Raises</dt> +<dd class="field-even"><p><strong>IndexError</strong> – If no partition exists</p> +</dd> +</dl> +</dd></dl> + +<dl class="method"> +<dt id="superset.jinja_context.PrestoTemplateProcessor.latest_partition"> +<code class="sig-name descname">latest_partition</code><span class="sig-paren">(</span><em class="sig-param">table_name</em><span class="sig-paren">)</span><a class="headerlink" href="#superset.jinja_context.PrestoTemplateProcessor.latest_partition" title="Permalink to this definition">¶</a></dt> +<dd><p>Gets the first value in the array of all latest partitions</p> +<dl class="field-list simple"> +<dt class="field-odd">Parameters</dt> +<dd class="field-odd"><p><strong>table_name</strong> (<code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code>) – table name in the format <cite>schema.table</cite></p> +</dd> +<dt class="field-even">Return type</dt> +<dd class="field-even"><p><code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code></p> +</dd> +<dt class="field-odd">Returns</dt> +<dd class="field-odd"><p>the first (or only) value in the latest partition array</p> +</dd> +<dt class="field-even">Raises</dt> +<dd class="field-even"><p><strong>IndexError</strong> – If no partition exists</p> +</dd> +</dl> +</dd></dl> + +<dl class="method"> +<dt id="superset.jinja_context.PrestoTemplateProcessor.latest_partitions"> +<code class="sig-name descname">latest_partitions</code><span class="sig-paren">(</span><em class="sig-param">table_name</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/superset/jinja_context.html#PrestoTemplateProcessor.latest_partitions"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#superset.jinja_context.PrestoTemplateProcessor.latest_partitions" title="Permalink to this definition">¶</a></dt> +<dd><p>Gets the array of all latest partitions</p> +<dl class="field-list simple"> +<dt class="field-odd">Parameters</dt> +<dd class="field-odd"><p><strong>table_name</strong> (<code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code>) – table name in the format <cite>schema.table</cite></p> +</dd> +<dt class="field-even">Return type</dt> +<dd class="field-even"><p><code class="xref py py-class docutils literal notranslate"><span class="pre">List</span></code>[<code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code>]</p> +</dd> +<dt class="field-odd">Returns</dt> +<dd class="field-odd"><p>the latest partition array</p> +</dd> +</dl> +</dd></dl> + </dd></dl> <dl class="class"> <dt id="superset.jinja_context.HiveTemplateProcessor"> -<em class="property">class </em><code class="sig-prename descclassname">superset.jinja_context.</code><code class="sig-name descname">HiveTemplateProcessor</code><span class="sig-paren">(</span><em class="sig-param">database=None</em>, <em class="sig-param">query=None</em>, <em class="sig-param">table=None</em>, <em class="sig-param">**kwargs</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/superset/jinja_context.html#HiveTemplateProcessor"><span class="vi [...] +<em class="property">class </em><code class="sig-prename descclassname">superset.jinja_context.</code><code class="sig-name descname">HiveTemplateProcessor</code><span class="sig-paren">(</span><em class="sig-param">database=None</em>, <em class="sig-param">query=None</em>, <em class="sig-param">table=None</em>, <em class="sig-param">extra_cache_keys=None</em>, <em class="sig-param">**kwargs</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/superset/jinja_c [...] <dd></dd></dl> </div>
