This is an automated email from the ASF dual-hosted git repository. rusackas pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-superset-site.git
commit 871fa2cc2a2403a6cc444b471c95efade24b3379 Author: Maxime Beauchemin <[email protected]> AuthorDate: Wed Sep 5 08:28:21 2018 -0700 Docs as of 0.28.0rc4 --- _images/big_number.png | Bin 10101 -> 103045 bytes _sources/faq.rst.txt | 60 ++++++++++++- _sources/index.rst.txt | 12 +++ _sources/installation.rst.txt | 134 ++++++++++++++++++++++++++++-- _static/img/loading.gif | Bin 16671 -> 79023 bytes _static/img/viz_thumbnails/big_number.png | Bin 10101 -> 103045 bytes faq.html | 54 +++++++++++- gallery.html | 2 +- index.html | 18 ++++ installation.html | 129 ++++++++++++++++++++++++++-- objects.inv | Bin 510 -> 545 bytes searchindex.js | 2 +- 12 files changed, 387 insertions(+), 24 deletions(-) diff --git a/_images/big_number.png b/_images/big_number.png index 01d6da4..90ac5a5 100644 Binary files a/_images/big_number.png and b/_images/big_number.png differ diff --git a/_sources/faq.rst.txt b/_sources/faq.rst.txt index 21e4e74..3b69044 100644 --- a/_sources/faq.rst.txt +++ b/_sources/faq.rst.txt @@ -114,8 +114,8 @@ never be affected by any dashboard level filtering. "filter_immune_slices": [324, 65, 92], "expanded_slices": {}, "filter_immune_slice_fields": { - "177": ["country_name", "__from", "__to"], - "32": ["__from", "__to"] + "177": ["country_name", "__time_range"], + "32": ["__time_range"] }, "timed_refresh_immune_slices": [324] } @@ -127,8 +127,8 @@ Now note the ``filter_immune_slice_fields`` key. This one allows you to be more specific and define for a specific slice_id, which filter fields should be disregarded. -Note the use of the ``__from`` and ``__to`` keywords, those are reserved -for dealing with the time boundary filtering mentioned above. +Note the use of the ``__time_range`` keyword, which is reserved for dealing +with the time boundary filtering mentioned above. But what happens with filtering when dealing with slices coming from different tables or databases? If the column name is shared, the filter will @@ -246,3 +246,55 @@ labels to colors in the ``JSON Metadata`` attribute using the "Boys": "#ADD8E6" } } + +Does Superset work with [insert database engine here]? +------------------------------------------------------ + +The community over time has curated a list of databases that work well with +Superset in the :ref:`ref_database_deps` section of the docs. Database +engines not listed in this page may work too. We rely on the +community to contribute to this knowledge base. + +.. _SQLAlchemy dialect: http://docs.sqlalchemy.org/en/latest/dialects/ +.. _DBAPI driver: https://www.python.org/dev/peps/pep-0249/ + +For a database engine to be supported in Superset through the +SQLAlchemy connector, it requires having a Python compliant +`SQLAlchemy dialect`_ as well as a +`DBAPI driver`_ defined. +Database that have limited SQL support may +work as well. For instance it's possible to connect +to Druid through the SQLAlchemy connector even though Druid does not support +joins and subqueries. Another key element for a database to be supported is through +the Superset `Database Engine Specification +<https://github.com/apache/incubator-superset/blob/master/superset/db_engine_specs.py>`_ +interface. This interface allows for defining database-specific configurations +and logic +that go beyond the SQLAlchemy and DBAPI scope. This includes features like: + + +* date-related SQL function that allow Superset to fetch different + time granularities when running time-series queries +* whether the engine supports subqueries. If false, Superset may run 2-phase + queries to compensate for the limitation +* methods around processing logs and inferring the percentage of completion + of a query +* technicalities as to how to handle cursors and connections if the driver + is not standard DBAPI +* more, read the code for more details + +Beyond the SQLAlchemy connector, it's also possible, though much more +involved, to extend Superset and write +your own connector. The only example of this at the moment is the Druid +connector, which is getting superseded by Druid's growing SQL support and +the recent availability of a DBAPI and SQLAlchemy driver. If the database +you are considering integrating has any kind of of SQL support, it's probably +preferable to go the SQLAlchemy route. Note that for a native connector to +be possible the database needs to have support for running OLAP-type queries +and should be able to things that are typical in basic SQL: + +- aggregate data +- apply filters (==, !=, >, <, >=, <=, IN, ...) +- apply HAVING-type filters +- be schema-aware, expose columns and types + diff --git a/_sources/index.rst.txt b/_sources/index.rst.txt index 370f516..54728eb 100644 --- a/_sources/index.rst.txt +++ b/_sources/index.rst.txt @@ -24,6 +24,18 @@ intelligence web application the code, it does indicate that the project has yet to be fully endorsed by the ASF. +Resources +========= +- `Superset's Github <https://github.com/apache/incubator-superset>`_, note + that `we use Github for issue tracking <https://github.com/apache/incubator-superset/issues>`_ +- Superset's + `contribution guidelines <https://github.com/apache/incubator-superset/blob/master/CONTRIBUTING.md>`_ + and + `code of conduct <https://github.com/apache/incubator-superset/blob/master/CODE_OF_CONDUCT.md>`_ + on Github. +- Our `mailing list archives <https://lists.apache.org/[email protected]>`_. + To subscribe, send an email to ``[email protected]`` + Overview ======== diff --git a/_sources/installation.rst.txt b/_sources/installation.rst.txt index 1f71ee2..7529323 100644 --- a/_sources/installation.rst.txt +++ b/_sources/installation.rst.txt @@ -4,7 +4,7 @@ Installation & Configuration Getting Started --------------- -Superset is tested against Python ``2.7`` and Python ``3.4``. +Superset is tested against Python ``2.7`` and Python ``3.6``. Airbnb currently uses 2.7.* in production. We do not plan on supporting Python ``2.6``. @@ -35,6 +35,30 @@ The Superset web server and the Superset Celery workers (optional) are stateless, so you can scale out by running on as many servers as needed. +Start with Docker +----------------- + +If you know docker, then you're lucky, we have shortcut road for you to +initialize development environment: :: + + git clone https://github.com/apache/incubator-superset/ + cd incubator-superset + cp contrib/docker/{docker-build.sh,docker-compose.yml,docker-entrypoint.sh,docker-init.sh,Dockerfile} . + cp contrib/docker/superset_config.py superset/ + bash -x docker-build.sh + docker-compose up -d + docker-compose exec superset bash + bash docker-init.sh + +After several minutes for superset initialization to finish, you can open +a browser and view `http://localhost:8088` to start your journey. + +If you are attempting to build on a Mac and it exits with 137 you need to increase your docker resources. +OSX instructions: https://docs.docker.com/docker-for-mac/#advanced (Search for memory) + +Or if you're curious and want to install superset from bottom up, then go +ahead. + OS dependencies --------------- @@ -53,7 +77,7 @@ the required dependencies are installed: :: sudo apt-get install build-essential libssl-dev libffi-dev python-dev python-pip libsasl2-dev libldap2-dev -**Ubuntu 16.04** If you have python3.5 installed alongside with python2.7, as is default on **Ubuntu 16.04 LTS**, run this command also +**Ubuntu 16.04** If you have python3.5 installed alongside with python2.7, as is default on **Ubuntu 16.04 LTS**, run this command also: :: sudo apt-get install build-essential libssl-dev libffi-dev python3.5-dev python-pip libsasl2-dev libldap2-dev @@ -275,6 +299,9 @@ auth postback endpoint, you can add them to *WTF_CSRF_EXEMPT_LIST* WTF_CSRF_EXEMPT_LIST = [''] + +.. _ref_database_deps: + Database dependencies --------------------- @@ -315,6 +342,8 @@ Here's a list of some of the recommended packages. +---------------+-------------------------------------+-------------------------------------------------+ | Athena | ``pip install "PyAthenaJDBC>1.0.9"``| ``awsathena+jdbc://`` | +---------------+-------------------------------------+-------------------------------------------------+ +| Athena | ``pip install "PyAthena>1.2.0"`` | ``awsathena+rest://`` | ++---------------+-------------------------------------+-------------------------------------------------+ | Vertica | ``pip install | ``vertica+vertica_python://`` | | | sqlalchemy-vertica-python`` | | +---------------+-------------------------------------+-------------------------------------------------+ @@ -342,6 +371,33 @@ Where you need to escape/encode at least the s3_staging_dir, i.e., :: s3://... -> s3%3A//... +You can also use `PyAthena` library(no java required) like this :: + + awsathena+rest://{aws_access_key_id}:{aws_secret_access_key}@athena.{region_name}.amazonaws.com/{schema_name}?s3_staging_dir={s3_staging_dir}&... + +See `PyAthena <https://github.com/laughingman7743/PyAthena#sqlalchemy>`_. + +Snowflake +--------- + +The connection string for Snowflake looks like this :: + + snowflake://{user}:{password}@{account}.{region}/{database}?role={role}&warehouse={warehouse} + +The schema is not necessary in the connection string, as it is defined per table/query. +The role and warehouse can be omitted if defaults are defined for the user, i.e. + + snowflake://{user}:{password}@{account}.{region}/{database} + +Make sure the user has privileges to access and use all required +databases/schemas/tables/views/warehouses, as the Snowflake SQLAlchemy engine does +not test for user rights during engine creation. + +*Note*: At the time of writing, there is a regression in the current stable version (1.1.2) of +snowflake-sqlalchemy package that causes problems when used with Superset. It is recommended to +use version 1.1.0 or try a newer version. + +See `Snowflake SQLAlchemy <https://github.com/snowflakedb/snowflake-sqlalchemy>`_. Caching ------- @@ -363,7 +419,7 @@ For setting your timeouts, this is done in the Superset metadata and goes up the "timeout searchpath", from your slice configuration, to your data source's configuration, to your database's and ultimately falls back into your global default defined in ``CACHE_CONFIG``. - + .. code-block:: python CACHE_CONFIG = { @@ -658,13 +714,81 @@ Note that it's also possible to implement you own logger by deriving Install Superset with helm in Kubernetes ---------------------------------------- -You can install Superset into Kubernetes with Helm <https://helm.sh/>. The chart is +You can install Superset into Kubernetes with Helm <https://helm.sh/>. The chart is located in ``install/helm``. To install Superset into your Kubernetes: .. code-block:: bash - helm upgrade --install superset ./install/helm/superset + helm upgrade --install superset ./install/helm/superset Note that the above command will install Superset into ``default`` namespace of your Kubernetes cluster. + +Custom OAuth2 configuration +--------------------------- + +Beyond FAB supported providers (github, twitter, linkedin, google, azure), its easy to connect Superset with other OAuth2 Authorization Server implementations that supports "code" authorization. + +The first step: Configure authorization in Superset ``superset_config.py``. + +.. code-block:: python + + AUTH_TYPE = AUTH_OAUTH + OAUTH_PROVIDERS = [ + { 'name':'egaSSO', + 'token_key':'access_token', # Name of the token in the response of access_token_url + 'icon':'fa-address-card', # Icon for the provider + 'remote_app': { + 'consumer_key':'myClientId', # Client Id (Identify Superset application) + 'consumer_secret':'MySecret', # Secret for this Client Id (Identify Superset application) + 'request_token_params':{ + 'scope': 'read' # Scope for the Authorization + }, + 'access_token_method':'POST', # HTTP Method to call access_token_url + 'access_token_params':{ # Additional parameters for calls to access_token_url + 'client_id':'myClientId' + }, + 'access_token_headers':{ # Additional headers for calls to access_token_url + 'Authorization': 'Basic Base64EncodedClientIdAndSecret' + }, + 'base_url':'https://myAuthorizationServer/oauth2AuthorizationServer/', + 'access_token_url':'https://myAuthorizationServer/oauth2AuthorizationServer/token', + 'authorize_url':'https://myAuthorizationServer/oauth2AuthorizationServer/authorize' + } + } + ] + + # Will allow user self registration, allowing to create Flask users from Authorized User + AUTH_USER_REGISTRATION = True + + # The default user self registration role + AUTH_USER_REGISTRATION_ROLE = "Public" + +Second step: Create a `CustomSsoSecurityManager` that extends `SupersetSecurityManager` and overrides `oauth_user_info`: + +.. code-block:: python + + from superset.security import SupersetSecurityManager + + class CustomSsoSecurityManager(SupersetSecurityManager): + + def oauth_user_info(self, provider, response=None): + logging.debug("Oauth2 provider: {0}.".format(provider)) + if provider == 'egaSSO': + # As example, this line request a GET to base_url + '/' + userDetails with Bearer Authentication, + # and expects that authorization server checks the token, and response with user details + me = self.appbuilder.sm.oauth_remotes[provider].get('userDetails').data + logging.debug("user_data: {0}".format(me)) + return { 'name' : me['name'], 'email' : me['email'], 'id' : me['user_name'], 'username' : me['user_name'], 'first_name':'', 'last_name':''} + ... + +This file must be located at the same directory than ``superset_config.py`` with the name ``custom_sso_security_manager.py``. + +Then we can add this two lines to ``superset_config.py``: + +.. code-block:: python + + from custom_sso_security_manager import CustomSsoSecurityManager + CUSTOM_SECURITY_MANAGER = CustomSsoSecurityManager + diff --git a/_static/img/loading.gif b/_static/img/loading.gif index 01ae393..d82fc5d 100644 Binary files a/_static/img/loading.gif and b/_static/img/loading.gif differ diff --git a/_static/img/viz_thumbnails/big_number.png b/_static/img/viz_thumbnails/big_number.png index 01d6da4..90ac5a5 100644 Binary files a/_static/img/viz_thumbnails/big_number.png and b/_static/img/viz_thumbnails/big_number.png differ diff --git a/faq.html b/faq.html index 4a1ad1e..405a2be 100644 --- a/faq.html +++ b/faq.html @@ -110,6 +110,7 @@ <li class="toctree-l2"><a class="reference internal" href="#how-can-i-set-a-default-filter-on-my-dashboard">How can I set a default filter on my dashboard?</a></li> <li class="toctree-l2"><a class="reference internal" href="#how-do-i-get-superset-to-refresh-the-schema-of-my-table">How do I get Superset to refresh the schema of my table?</a></li> <li class="toctree-l2"><a class="reference internal" href="#is-there-a-way-to-force-the-use-specific-colors">Is there a way to force the use specific colors?</a></li> +<li class="toctree-l2"><a class="reference internal" href="#does-superset-work-with-insert-database-engine-here">Does Superset work with [insert database engine here]?</a></li> </ul> </li> </ul> @@ -260,8 +261,8 @@ never be affected by any dashboard level filtering.</p> <span class="nt">"filter_immune_slices"</span><span class="p">:</span> <span class="p">[</span><span class="mi">324</span><span class="p">,</span> <span class="mi">65</span><span class="p">,</span> <span class="mi">92</span><span class="p">],</span> <span class="nt">"expanded_slices"</span><span class="p">:</span> <span class="p">{},</span> <span class="nt">"filter_immune_slice_fields"</span><span class="p">:</span> <span class="p">{</span> - <span class="nt">"177"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"country_name"</span><span class="p">,</span> <span class="s2">"__from"</span><span class="p">,</span> <span class="s2">"__to"</span><span class="p">],</span> - <span class="nt">"32"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"__from"</span><span class="p">,</span> <span class="s2">"__to"</span><span class="p">]</span> + <span class="nt">"177"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"country_name"</span><span class="p">,</span> <span class="s2">"__time_range"</span><span class="p">],</span> + <span class="nt">"32"</span><span class="p">:</span> <span class="p">[</span><span class="s2">"__time_range"</span><span class="p">]</span> <span class="p">},</span> <span class="nt">"timed_refresh_immune_slices"</span><span class="p">:</span> <span class="p">[</span><span class="mi">324</span><span class="p">]</span> <span class="p">}</span> @@ -272,8 +273,8 @@ dashboard level filtering.</p> <p>Now note the <code class="docutils literal notranslate"><span class="pre">filter_immune_slice_fields</span></code> key. This one allows you to be more specific and define for a specific slice_id, which filter fields should be disregarded.</p> -<p>Note the use of the <code class="docutils literal notranslate"><span class="pre">__from</span></code> and <code class="docutils literal notranslate"><span class="pre">__to</span></code> keywords, those are reserved -for dealing with the time boundary filtering mentioned above.</p> +<p>Note the use of the <code class="docutils literal notranslate"><span class="pre">__time_range</span></code> keyword, which is reserved for dealing +with the time boundary filtering mentioned above.</p> <p>But what happens with filtering when dealing with slices coming from different tables or databases? If the column name is shared, the filter will be applied, it’s as simple as that.</p> @@ -370,6 +371,51 @@ labels to colors in the <code class="docutils literal notranslate"><span class=" </pre></div> </div> </div> +<div class="section" id="does-superset-work-with-insert-database-engine-here"> +<h2>Does Superset work with [insert database engine here]?<a class="headerlink" href="#does-superset-work-with-insert-database-engine-here" title="Permalink to this headline">¶</a></h2> +<p>The community over time has curated a list of databases that work well with +Superset in the <a class="reference internal" href="installation.html#ref-database-deps"><span class="std std-ref">Database dependencies</span></a> section of the docs. Database +engines not listed in this page may work too. We rely on the +community to contribute to this knowledge base.</p> +<p>For a database engine to be supported in Superset through the +SQLAlchemy connector, it requires having a Python compliant +<a class="reference external" href="http://docs.sqlalchemy.org/en/latest/dialects/";>SQLAlchemy dialect</a> as well as a +<a class="reference external" href="https://www.python.org/dev/peps/pep-0249/";>DBAPI driver</a> defined. +Database that have limited SQL support may +work as well. For instance it’s possible to connect +to Druid through the SQLAlchemy connector even though Druid does not support +joins and subqueries. Another key element for a database to be supported is through +the Superset <a class="reference external" href="https://github.com/apache/incubator-superset/blob/master/superset/db_engine_specs.py";>Database Engine Specification</a> +interface. This interface allows for defining database-specific configurations +and logic +that go beyond the SQLAlchemy and DBAPI scope. This includes features like:</p> +<ul class="simple"> +<li>date-related SQL function that allow Superset to fetch different +time granularities when running time-series queries</li> +<li>whether the engine supports subqueries. If false, Superset may run 2-phase +queries to compensate for the limitation</li> +<li>methods around processing logs and inferring the percentage of completion +of a query</li> +<li>technicalities as to how to handle cursors and connections if the driver +is not standard DBAPI</li> +<li>more, read the code for more details</li> +</ul> +<p>Beyond the SQLAlchemy connector, it’s also possible, though much more +involved, to extend Superset and write +your own connector. The only example of this at the moment is the Druid +connector, which is getting superseded by Druid’s growing SQL support and +the recent availability of a DBAPI and SQLAlchemy driver. If the database +you are considering integrating has any kind of of SQL support, it’s probably +preferable to go the SQLAlchemy route. Note that for a native connector to +be possible the database needs to have support for running OLAP-type queries +and should be able to things that are typical in basic SQL:</p> +<ul class="simple"> +<li>aggregate data</li> +<li>apply filters (==, !=, >, <, >=, <=, IN, …)</li> +<li>apply HAVING-type filters</li> +<li>be schema-aware, expose columns and types</li> +</ul> +</div> </div> diff --git a/gallery.html b/gallery.html index bcf5f6e..f544fc8 100644 --- a/gallery.html +++ b/gallery.html @@ -160,7 +160,7 @@ <h1>Visualizations Gallery<a class="headerlink" href="#visualizations-gallery" title="Permalink to this headline">¶</a></h1> <a class="reference internal image-reference" href="_images/area.png"><img alt="_images/area.png" src="_images/area.png" style="width: 128.0px; height: 128.0px;" /></a> <a class="reference internal image-reference" href="_images/bar.png"><img alt="_images/bar.png" src="_images/bar.png" style="width: 128.0px; height: 128.0px;" /></a> -<a class="reference internal image-reference" href="_images/big_number.png"><img alt="_images/big_number.png" src="_images/big_number.png" style="width: 128.0px; height: 128.0px;" /></a> +<a class="reference internal image-reference" href="_images/big_number.png"><img alt="_images/big_number.png" src="_images/big_number.png" style="width: 270.0px; height: 270.0px;" /></a> <a class="reference internal image-reference" href="_images/big_number_total.png"><img alt="_images/big_number_total.png" src="_images/big_number_total.png" style="width: 128.0px; height: 128.0px;" /></a> <a class="reference internal image-reference" href="_images/box_plot.png"><img alt="_images/box_plot.png" src="_images/box_plot.png" style="width: 128.0px; height: 128.0px;" /></a> <a class="reference internal image-reference" href="_images/bubble.png"><img alt="_images/bubble.png" src="_images/bubble.png" style="width: 128.0px; height: 128.0px;" /></a> diff --git a/index.html b/index.html index c475bd0..90beee3 100644 --- a/index.html +++ b/index.html @@ -178,6 +178,20 @@ necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p> </div> +<div class="section" id="resources"> +<h2>Resources<a class="headerlink" href="#resources" title="Permalink to this headline">¶</a></h2> +<ul class="simple"> +<li><a class="reference external" href="https://github.com/apache/incubator-superset";>Superset’s Github</a>, note +that <a class="reference external" href="https://github.com/apache/incubator-superset/issues";>we use Github for issue tracking</a></li> +<li>Superset’s +<a class="reference external" href="https://github.com/apache/incubator-superset/blob/master/CONTRIBUTING.md";>contribution guidelines</a> +and +<a class="reference external" href="https://github.com/apache/incubator-superset/blob/master/CODE_OF_CONDUCT.md";>code of conduct</a> +on Github.</li> +<li>Our <a class="reference external" href="https://lists.apache.org/list.html?dev@superset.apache.org";>mailing list archives</a>. +To subscribe, send an email to <code class="docutils literal notranslate"><span class="pre">dev-subscribe@superset.apache.org</span></code></li> +</ul> +</div> <div class="section" id="overview"> <h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline">¶</a></h2> <div class="section" id="features"> @@ -217,6 +231,7 @@ to the user</li> <li class="toctree-l1"><a class="reference internal" href="installation.html">Installation & Configuration</a><ul> <li class="toctree-l2"><a class="reference internal" href="installation.html#getting-started">Getting Started</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#cloud-native">Cloud-native!</a></li> +<li class="toctree-l2"><a class="reference internal" href="installation.html#start-with-docker">Start with Docker</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#os-dependencies">OS dependencies</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#python-virtualenv">Python virtualenv</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#python-s-setup-tools-and-pip">Python’s setup tools and pip</a></li> @@ -227,6 +242,7 @@ to the user</li> <li class="toctree-l2"><a class="reference internal" href="installation.html#configuration">Configuration</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#database-dependencies">Database dependencies</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#aws-athena">(AWS) Athena</a></li> +<li class="toctree-l2"><a class="reference internal" href="installation.html#snowflake">Snowflake</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#caching">Caching</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#deeper-sqlalchemy-integration">Deeper SQLAlchemy integration</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#schemas-postgres-redshift">Schemas (Postgres & Redshift)</a></li> @@ -241,6 +257,7 @@ to the user</li> <li class="toctree-l2"><a class="reference internal" href="installation.html#blueprints">Blueprints</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#statsd-logging">StatsD logging</a></li> <li class="toctree-l2"><a class="reference internal" href="installation.html#install-superset-with-helm-in-kubernetes">Install Superset with helm in Kubernetes</a></li> +<li class="toctree-l2"><a class="reference internal" href="installation.html#custom-oauth2-configuration">Custom OAuth2 configuration</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial - Creating your first dashboard</a><ul> @@ -292,6 +309,7 @@ to the user</li> <li class="toctree-l2"><a class="reference internal" href="faq.html#how-can-i-set-a-default-filter-on-my-dashboard">How can I set a default filter on my dashboard?</a></li> <li class="toctree-l2"><a class="reference internal" href="faq.html#how-do-i-get-superset-to-refresh-the-schema-of-my-table">How do I get Superset to refresh the schema of my table?</a></li> <li class="toctree-l2"><a class="reference internal" href="faq.html#is-there-a-way-to-force-the-use-specific-colors">Is there a way to force the use specific colors?</a></li> +<li class="toctree-l2"><a class="reference internal" href="faq.html#does-superset-work-with-insert-database-engine-here">Does Superset work with [insert database engine here]?</a></li> </ul> </li> </ul> diff --git a/installation.html b/installation.html index e52db5c..abc4769 100644 --- a/installation.html +++ b/installation.html @@ -90,6 +90,7 @@ <li class="toctree-l1 current"><a class="current reference internal" href="#">Installation & Configuration</a><ul> <li class="toctree-l2"><a class="reference internal" href="#getting-started">Getting Started</a></li> <li class="toctree-l2"><a class="reference internal" href="#cloud-native">Cloud-native!</a></li> +<li class="toctree-l2"><a class="reference internal" href="#start-with-docker">Start with Docker</a></li> <li class="toctree-l2"><a class="reference internal" href="#os-dependencies">OS dependencies</a></li> <li class="toctree-l2"><a class="reference internal" href="#python-virtualenv">Python virtualenv</a></li> <li class="toctree-l2"><a class="reference internal" href="#python-s-setup-tools-and-pip">Python’s setup tools and pip</a></li> @@ -100,6 +101,7 @@ <li class="toctree-l2"><a class="reference internal" href="#configuration">Configuration</a></li> <li class="toctree-l2"><a class="reference internal" href="#database-dependencies">Database dependencies</a></li> <li class="toctree-l2"><a class="reference internal" href="#aws-athena">(AWS) Athena</a></li> +<li class="toctree-l2"><a class="reference internal" href="#snowflake">Snowflake</a></li> <li class="toctree-l2"><a class="reference internal" href="#caching">Caching</a></li> <li class="toctree-l2"><a class="reference internal" href="#deeper-sqlalchemy-integration">Deeper SQLAlchemy integration</a></li> <li class="toctree-l2"><a class="reference internal" href="#schemas-postgres-redshift">Schemas (Postgres & Redshift)</a></li> @@ -114,6 +116,7 @@ <li class="toctree-l2"><a class="reference internal" href="#blueprints">Blueprints</a></li> <li class="toctree-l2"><a class="reference internal" href="#statsd-logging">StatsD logging</a></li> <li class="toctree-l2"><a class="reference internal" href="#install-superset-with-helm-in-kubernetes">Install Superset with helm in Kubernetes</a></li> +<li class="toctree-l2"><a class="reference internal" href="#custom-oauth2-configuration">Custom OAuth2 configuration</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial - Creating your first dashboard</a></li> @@ -188,7 +191,7 @@ <h1>Installation & Configuration<a class="headerlink" href="#installation-configuration" title="Permalink to this headline">¶</a></h1> <div class="section" id="getting-started"> <h2>Getting Started<a class="headerlink" href="#getting-started" title="Permalink to this headline">¶</a></h2> -<p>Superset is tested against Python <code class="docutils literal notranslate"><span class="pre">2.7</span></code> and Python <code class="docutils literal notranslate"><span class="pre">3.4</span></code>. +<p>Superset is tested against Python <code class="docutils literal notranslate"><span class="pre">2.7</span></code> and Python <code class="docutils literal notranslate"><span class="pre">3.6</span></code>. Airbnb currently uses 2.7.* in production. We do not plan on supporting Python <code class="docutils literal notranslate"><span class="pre">2.6</span></code>.</p> </div> @@ -216,6 +219,27 @@ day.</p> are stateless, so you can scale out by running on as many servers as needed.</p> </div> +<div class="section" id="start-with-docker"> +<h2>Start with Docker<a class="headerlink" href="#start-with-docker" title="Permalink to this headline">¶</a></h2> +<p>If you know docker, then you’re lucky, we have shortcut road for you to +initialize development environment:</p> +<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">git</span> <span class="n">clone</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">apache</span><span class="o">/</span><span class="n">incubator</span><span class="o">-</span><span class="n">superset</span><span class="o">/</span> +<span class="n">cd</span> <span class="n">incubator</span><span class="o">-</span><span class="n">superset</span> +<span class="n">cp</span> <span class="n">contrib</span><span class="o">/</span><span class="n">docker</span><span class="o">/</span><span class="p">{</span><span class="n">docker</span><span class="o">-</span><span class="n">build</span><span class="o">.</span><span class="n">sh</span><span class="p">,</span><span class="n">docker</span><span class="o">-</span><span class="n">compose</span><span class="o">.</span><span class="n">yml</span><span class="p">,</span><span class="n">docker</ [...] +<span class="n">cp</span> <span class="n">contrib</span><span class="o">/</span><span class="n">docker</span><span class="o">/</span><span class="n">superset_config</span><span class="o">.</span><span class="n">py</span> <span class="n">superset</span><span class="o">/</span> +<span class="n">bash</span> <span class="o">-</span><span class="n">x</span> <span class="n">docker</span><span class="o">-</span><span class="n">build</span><span class="o">.</span><span class="n">sh</span> +<span class="n">docker</span><span class="o">-</span><span class="n">compose</span> <span class="n">up</span> <span class="o">-</span><span class="n">d</span> +<span class="n">docker</span><span class="o">-</span><span class="n">compose</span> <span class="n">exec</span> <span class="n">superset</span> <span class="n">bash</span> +<span class="n">bash</span> <span class="n">docker</span><span class="o">-</span><span class="n">init</span><span class="o">.</span><span class="n">sh</span> +</pre></div> +</div> +<p>After several minutes for superset initialization to finish, you can open +a browser and view <cite>http://localhost:8088</cite> to start your journey.</p> +<p>If you are attempting to build on a Mac and it exits with 137 you need to increase your docker resources. +OSX instructions: <a class="reference external" href="https://docs.docker.com/docker-for-mac/#advanced";>https://docs.docker.com/docker-for-mac/#advanced</a> (Search for memory)</p> +<p>Or if you’re curious and want to install superset from bottom up, then go +ahead.</p> +</div> <div class="section" id="os-dependencies"> <h2>OS dependencies<a class="headerlink" href="#os-dependencies" title="Permalink to this headline">¶</a></h2> <p>Superset stores database connection information in its metadata database. @@ -230,9 +254,10 @@ the required dependencies are installed:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">install</span> <span class="n">build</span><span class="o">-</span><span class="n">essential</span> <span class="n">libssl</span><span class="o">-</span><span class="n">dev</span> <span class="n">libffi</span><span class="o">-</span><span class="n">dev</span> <span class="n">python</spa [...] </pre></div> </div> -<p><strong>Ubuntu 16.04</strong> If you have python3.5 installed alongside with python2.7, as is default on <strong>Ubuntu 16.04 LTS</strong>, run this command also</p> -<blockquote> -<div>sudo apt-get install build-essential libssl-dev libffi-dev python3.5-dev python-pip libsasl2-dev libldap2-dev</div></blockquote> +<p><strong>Ubuntu 16.04</strong> If you have python3.5 installed alongside with python2.7, as is default on <strong>Ubuntu 16.04 LTS</strong>, run this command also:</p> +<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">install</span> <span class="n">build</span><span class="o">-</span><span class="n">essential</span> <span class="n">libssl</span><span class="o">-</span><span class="n">dev</span> <span class="n">libffi</span><span class="o">-</span><span class="n">dev</span> <span class="n">python3</sp [...] +</pre></div> +</div> <p>otherwhise build for <code class="docutils literal notranslate"><span class="pre">cryptography</span></code> fails.</p> <p>For <strong>Fedora</strong> and <strong>RHEL-derivatives</strong>, the following command will ensure that the required dependencies are installed:</p> @@ -440,7 +465,7 @@ auth postback endpoint, you can add them to <em>WTF_CSRF_EXEMPT_LIST</em></p> <div>WTF_CSRF_EXEMPT_LIST = [‘’]</div></blockquote> </div> <div class="section" id="database-dependencies"> -<h2>Database dependencies<a class="headerlink" href="#database-dependencies" title="Permalink to this headline">¶</a></h2> +<span id="ref-database-deps"></span><h2>Database dependencies<a class="headerlink" href="#database-dependencies" title="Permalink to this headline">¶</a></h2> <p>Superset does not ship bundled with connectivity to databases, except for Sqlite, which is part of the Python standard library. You’ll need to install the required packages for the database you @@ -512,21 +537,25 @@ connect to the databases you want to access through Superset.</p> <td><code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">"PyAthenaJDBC>1.0.9"</span></code></td> <td><code class="docutils literal notranslate"><span class="pre">awsathena+jdbc://</span></code></td> </tr> -<tr class="row-odd"><td>Vertica</td> +<tr class="row-odd"><td>Athena</td> +<td><code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">"PyAthena>1.2.0"</span></code></td> +<td><code class="docutils literal notranslate"><span class="pre">awsathena+rest://</span></code></td> +</tr> +<tr class="row-even"><td>Vertica</td> <td><code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">sqlalchemy-vertica-python</span></code></td> <td><code class="docutils literal notranslate"><span class="pre">vertica+vertica_python://</span></code></td> </tr> -<tr class="row-even"><td>ClickHouse</td> +<tr class="row-odd"><td>ClickHouse</td> <td><code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">sqlalchemy-clickhouse</span></code></td> <td><code class="docutils literal notranslate"><span class="pre">clickhouse://</span></code></td> </tr> -<tr class="row-odd"><td>Kylin</td> +<tr class="row-even"><td>Kylin</td> <td><code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">kylinpy</span></code></td> <td><code class="docutils literal notranslate"><span class="pre">kylin://</span></code></td> </tr> -<tr class="row-even"><td>BigQuery</td> +<tr class="row-odd"><td>BigQuery</td> <td><code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">pybigquery</span></code></td> <td><code class="docutils literal notranslate"><span class="pre">bigquery://</span></code></td> </tr> @@ -547,6 +576,29 @@ database you want to connect to should get you to the right place.</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">s3</span><span class="p">:</span><span class="o">//...</span> <span class="o">-></span> <span class="n">s3</span><span class="o">%</span><span class="mi">3</span><span class="n">A</span><span class="o">//...</span> </pre></div> </div> +<p>You can also use <cite>PyAthena</cite> library(no java required) like this</p> +<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>awsathena+rest://{aws_access_key_id}:{aws_secret_access_key}@athena.{region_name}.amazonaws.com/{schema_name}?s3_staging_dir={s3_staging_dir}&... +</pre></div> +</div> +<p>See <a class="reference external" href="https://github.com/laughingman7743/PyAthena#sqlalchemy";>PyAthena</a>.</p> +</div> +<div class="section" id="snowflake"> +<h2>Snowflake<a class="headerlink" href="#snowflake" title="Permalink to this headline">¶</a></h2> +<p>The connection string for Snowflake looks like this</p> +<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>snowflake://{user}:{password}@{account}.{region}/{database}?role={role}&warehouse={warehouse} +</pre></div> +</div> +<p>The schema is not necessary in the connection string, as it is defined per table/query. +The role and warehouse can be omitted if defaults are defined for the user, i.e.</p> +<blockquote> +<div>snowflake://{user}:{password}@{account}.{region}/{database}</div></blockquote> +<p>Make sure the user has privileges to access and use all required +databases/schemas/tables/views/warehouses, as the Snowflake SQLAlchemy engine does +not test for user rights during engine creation.</p> +<p><em>Note</em>: At the time of writing, there is a regression in the current stable version (1.1.2) of +snowflake-sqlalchemy package that causes problems when used with Superset. It is recommended to +use version 1.1.0 or try a newer version.</p> +<p>See <a class="reference external" href="https://github.com/snowflakedb/snowflake-sqlalchemy";>Snowflake SQLAlchemy</a>.</p> </div> <div class="section" id="caching"> <h2>Caching<a class="headerlink" href="#caching" title="Permalink to this headline">¶</a></h2> @@ -831,6 +883,65 @@ located in <code class="docutils literal notranslate"><span class="pre">install/ </div> <p>Note that the above command will install Superset into <code class="docutils literal notranslate"><span class="pre">default</span></code> namespace of your Kubernetes cluster.</p> </div> +<div class="section" id="custom-oauth2-configuration"> +<h2>Custom OAuth2 configuration<a class="headerlink" href="#custom-oauth2-configuration" title="Permalink to this headline">¶</a></h2> +<p>Beyond FAB supported providers (github, twitter, linkedin, google, azure), its easy to connect Superset with other OAuth2 Authorization Server implementations that supports “code” authorization.</p> +<p>The first step: Configure authorization in Superset <code class="docutils literal notranslate"><span class="pre">superset_config.py</span></code>.</p> +<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">AUTH_TYPE</span> <span class="o">=</span> <span class="n">AUTH_OAUTH</span> +<span class="n">OAUTH_PROVIDERS</span> <span class="o">=</span> <span class="p">[</span> + <span class="p">{</span> <span class="s1">'name'</span><span class="p">:</span><span class="s1">'egaSSO'</span><span class="p">,</span> + <span class="s1">'token_key'</span><span class="p">:</span><span class="s1">'access_token'</span><span class="p">,</span> <span class="c1"># Name of the token in the response of access_token_url</span> + <span class="s1">'icon'</span><span class="p">:</span><span class="s1">'fa-address-card'</span><span class="p">,</span> <span class="c1"># Icon for the provider</span> + <span class="s1">'remote_app'</span><span class="p">:</span> <span class="p">{</span> + <span class="s1">'consumer_key'</span><span class="p">:</span><span class="s1">'myClientId'</span><span class="p">,</span> <span class="c1"># Client Id (Identify Superset application)</span> + <span class="s1">'consumer_secret'</span><span class="p">:</span><span class="s1">'MySecret'</span><span class="p">,</span> <span class="c1"># Secret for this Client Id (Identify Superset application)</span> + <span class="s1">'request_token_params'</span><span class="p">:{</span> + <span class="s1">'scope'</span><span class="p">:</span> <span class="s1">'read'</span> <span class="c1"># Scope for the Authorization</span> + <span class="p">},</span> + <span class="s1">'access_token_method'</span><span class="p">:</span><span class="s1">'POST'</span><span class="p">,</span> <span class="c1"># HTTP Method to call access_token_url</span> + <span class="s1">'access_token_params'</span><span class="p">:{</span> <span class="c1"># Additional parameters for calls to access_token_url</span> + <span class="s1">'client_id'</span><span class="p">:</span><span class="s1">'myClientId'</span> + <span class="p">},</span> + <span class="s1">'access_token_headers'</span><span class="p">:{</span> <span class="c1"># Additional headers for calls to access_token_url</span> + <span class="s1">'Authorization'</span><span class="p">:</span> <span class="s1">'Basic Base64EncodedClientIdAndSecret'</span> + <span class="p">},</span> + <span class="s1">'base_url'</span><span class="p">:</span><span class="s1">'https://myAuthorizationServer/oauth2AuthorizationServer/'</span><span class="p">,</span> + <span class="s1">'access_token_url'</span><span class="p">:</span><span class="s1">'https://myAuthorizationServer/oauth2AuthorizationServer/token'</span><span class="p">,</span> + <span class="s1">'authorize_url'</span><span class="p">:</span><span class="s1">'https://myAuthorizationServer/oauth2AuthorizationServer/authorize'</span> + <span class="p">}</span> + <span class="p">}</span> +<span class="p">]</span> + +<span class="c1"># Will allow user self registration, allowing to create Flask users from Authorized User</span> +<span class="n">AUTH_USER_REGISTRATION</span> <span class="o">=</span> <span class="bp">True</span> + +<span class="c1"># The default user self registration role</span> +<span class="n">AUTH_USER_REGISTRATION_ROLE</span> <span class="o">=</span> <span class="s2">"Public"</span> +</pre></div> +</div> +<p>Second step: Create a <cite>CustomSsoSecurityManager</cite> that extends <cite>SupersetSecurityManager</cite> and overrides <cite>oauth_user_info</cite>:</p> +<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">superset.security</span> <span class="kn">import</span> <span class="n">SupersetSecurityManager</span> + +<span class="k">class</span> <span class="nc">CustomSsoSecurityManager</span><span class="p">(</span><span class="n">SupersetSecurityManager</span><span class="p">):</span> + + <span class="k">def</span> <span class="nf">oauth_user_info</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">provider</span><span class="p">,</span> <span class="n">response</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span> + <span class="n">logging</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Oauth2 provider: {0}."</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">provider</span><span class="p">))</span> + <span class="k">if</span> <span class="n">provider</span> <span class="o">==</span> <span class="s1">'egaSSO'</span><span class="p">:</span> + <span class="c1"># As example, this line request a GET to base_url + '/' + userDetails with Bearer Authentication,</span> + <span class="c1"># and expects that authorization server checks the token, and response with user details</span> + <span class="n">me</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">appbuilder</span><span class="o">.</span><span class="n">sm</span><span class="o">.</span><span class="n">oauth_remotes</span><span class="p">[</span><span class="n">provider</span><span class="p">]</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'userDetails'</span><span class="p">)</span><span class=" [...] + <span class="n">logging</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"user_data: {0}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">me</span><span class="p">))</span> + <span class="k">return</span> <span class="p">{</span> <span class="s1">'name'</span> <span class="p">:</span> <span class="n">me</span><span class="p">[</span><span class="s1">'name'</span><span class="p">],</span> <span class="s1">'email'</span> <span class="p">:</span> <span class="n">me</span><span class="p">[</span><span class="s1">'email'</span><span class="p">],</span> <span class="s1">'id'</span> <span class="p">:</span> <span c [...] + <span class="o">...</span> +</pre></div> +</div> +<p>This file must be located at the same directory than <code class="docutils literal notranslate"><span class="pre">superset_config.py</span></code> with the name <code class="docutils literal notranslate"><span class="pre">custom_sso_security_manager.py</span></code>.</p> +<p>Then we can add this two lines to <code class="docutils literal notranslate"><span class="pre">superset_config.py</span></code>:</p> +<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">custom_sso_security_manager</span> <span class="kn">import</span> <span class="n">CustomSsoSecurityManager</span> +<span class="n">CUSTOM_SECURITY_MANAGER</span> <span class="o">=</span> <span class="n">CustomSsoSecurityManager</span> +</pre></div> +</div> +</div> </div> diff --git a/objects.inv b/objects.inv index b6a8f76..4180264 100644 Binary files a/objects.inv and b/objects.inv differ diff --git a/searchindex.js b/searchindex.js index 29d374c..d0b7b25 100644 --- a/searchindex.js +++ b/searchindex.js @@ -1 +1 @@ -Search.setIndex({docnames:["druid","faq","gallery","import_export_datasources","index","installation","misc","security","sqllab","tutorial","videos","visualization"],envversion:53,filenames:["druid.rst","faq.rst","gallery.rst","import_export_datasources.rst","index.rst","installation.rst","misc.rst","security.rst","sqllab.rst","tutorial.rst","videos.rst","visualization.rst"],objects:{"superset.jinja_context":{PrestoTemplateProcessor:[8,0,1,""],url_param:[8,1,1,""]}},objnames:{"0":["py"," [...] \ No newline at end of file +Search.setIndex({docnames:["druid","faq","gallery","import_export_datasources","index","installation","misc","security","sqllab","tutorial","videos","visualization"],envversion:53,filenames:["druid.rst","faq.rst","gallery.rst","import_export_datasources.rst","index.rst","installation.rst","misc.rst","security.rst","sqllab.rst","tutorial.rst","videos.rst","visualization.rst"],objects:{"superset.jinja_context":{PrestoTemplateProcessor:[8,0,1,""],url_param:[8,1,1,""]}},objnames:{"0":["py"," [...] \ No newline at end of file
