[GitHub] [incubator-superset] betodealmeida opened a new pull request #7416: Scheduling queries from SQL Lab

Tue, 30 Apr 2019 13:36:40 -0700 

betodealmeida opened a new pull request #7416: Scheduling queries from SQL Lab
URL: https://github.com/apache/incubator-superset/pull/7416
 
 
   ### CATEGORY
   
   Choose one
   
   - [ ] Bug Fix
   - [X] Enhancement (new features, refinement)
   - [ ] Refactor
   - [ ] Add tests
   - [ ] Build / Development Environment
   - [ ] Documentation
   
   ### SUMMARY
   <!--- Describe the change below, including rationale and design decisions -->
   
   This PR introduces a lightweight way of scheduling queries in SQL Lab. If 
the feature flag `SCHEDULED_QUERIES` is enabled with proper configuration, a 
button called "Schedule Query" will show up in SQL Lab. The button allows 
queries to be saved with extra metadata that allows an external scheduler to 
run it periodically by polling the `/savedqueryviewapi/api/read` endpoint.
   
   The sample configuration can be changed or expanded to support different 
metadata needed, depending on the scheduler. We tested it with [Apache 
Airflow](https://airflow.apache.org/) at Lyft successfully.
   
   ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
   <!--- Skip this if not applicable -->
   
   <img width="1698" alt="Screen Shot 2019-04-30 at 1 31 14 PM" 
src="https://user-images.githubusercontent.com/1534870/56991363-55a3de80-6b4c-11e9-848b-71b6348ff378.png";>
   <img width="1698" alt="Screen Shot 2019-04-30 at 1 31 46 PM" 
src="https://user-images.githubusercontent.com/1534870/56991381-59cffc00-6b4c-11e9-8f93-0da899648174.png";>
   
   This will generate the following payload in `savedqueryviewapi/api/read`:
   
   ```json
   {
     ...
     "result": [
       ...
       {
         "description": null, 
         "extra_json": 
"{\"schedule_info\":{\"output_table\":\"beto_output_table\",\"start_date\":\"2019-04-30T16:00:00.000Z\",\"end_date\":\"2019-05-30T16:00:00.000Z\",\"schedule_interval\":\"@daily\",\"dependencies\":[\"hive://schema.table\"]}}",
 
         "id": 4, 
         "label": "Untitled Query", 
         "schema": "null", 
         "sql": "SELECT *\nFROM\nWHERE", 
         "sqlalchemy_uri": 
"sqlite:////Users/bdealmeida/.superset/unittests.db", 
         "user_email": "[email protected]"
       }
     ],
   }
   ```
   
   ### TEST PLAN
   <!--- What steps should be taken to verify the changes -->
   
   Tested end-to-end at Lyft with a Hive query.
   
   ### ADDITIONAL INFORMATION
   <!--- Check any relevant boxes with "x" -->
   <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
   - [ ] Has associated issue:
   - [X] Changes UI
   - [ ] Requires DB Migration.
   - [ ] Confirm DB Migration upgrade and downgrade tested.
   - [X] Introduces new feature or API
   - [ ] Removes existing feature or API
   
   ### REVIEWERS
   
   @mistercrunch @datability-io @DiggidyDave @khtruong @argentfalcon

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to