[GitHub] [incubator-superset] etr2460 opened a new pull request #9284: Reduce dashboard bootstrap payload

Wed, 11 Mar 2020 11:53:15 -0700 

etr2460 opened a new pull request #9284: Reduce dashboard bootstrap payload
URL: https://github.com/apache/incubator-superset/pull/9284
 
 
   ### CATEGORY
   
   Choose one
   
   - [ ] Bug Fix
   - [x] Enhancement (new features, refinement)
   - [ ] Refactor
   - [ ] Add tests
   - [ ] Build / Development Environment
   - [ ] Documentation
   
   ### SUMMARY
   Currently, when loading a dashboard we send down the complete data payload 
for all datasources used within the dashboard. This is somewhat of a waste, 
because in the dashboard view the entire datasource isn't needed, only the 
pieces required to render the slices within the dashboard (as opposed to the 
explore view, where we need the entire datasource). When datasources have 
hundreds or thousands of columns or metrics, this can get out of hand quite 
quickly; we have a datasource that ends up being about 6MB of uncompressed JSON.
   
   This PR adds a new function to the datasource model that filters the full 
data to only pieces required for a set of slices on a dashboard. This consists 
of:
   - Removing unneeded columns
   - Removing unneeded metrics
   - Removing unneeded columns and metrics from the verbose_map
   - Removing the datasource description
   
   When determining what columns and metrics were required to render charts 
properly, I referenced the form_data definition from CONTRIBUTING.md to ensure 
that we included all the required columns and metrics for all the various 
charts.
   
   ### TEST PLAN
   New unit tests. Also load the world bank dashboard before and after the 
change. See it render the same way, but require significantly less data
   
   Before:
   <img width="382" alt="Screen Shot 2020-03-11 at 11 44 56 AM" 
src="https://user-images.githubusercontent.com/7409244/76452568-96849a80-638e-11ea-848d-48e3a4d130a2.png";>
   <img width="1920" alt="Screen Shot 2020-03-11 at 11 45 10 AM" 
src="https://user-images.githubusercontent.com/7409244/76452573-971d3100-638e-11ea-936e-8298e1e381ff.png";>
   
   After:
   <img width="387" alt="Screen Shot 2020-03-11 at 11 43 43 AM" 
src="https://user-images.githubusercontent.com/7409244/76452596-a00e0280-638e-11ea-8501-2980ef9d2c8c.png";>
   <img width="1920" alt="Screen Shot 2020-03-11 at 11 44 10 AM" 
src="https://user-images.githubusercontent.com/7409244/76452603-a0a69900-638e-11ea-9e15-0d8dd91fe3fc.png";>
   
   
   ### ADDITIONAL INFORMATION
   <!--- Check any relevant boxes with "x" -->
   <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
   - [ ] Has associated issue:
   - [ ] Changes UI
   - [ ] Requires DB Migration.
   - [ ] Confirm DB Migration upgrade and downgrade tested.
   - [ ] Introduces new feature or API
   - [ ] Removes existing feature or API
   
   ### REVIEWERS
   to: @john-bodley @villebro @mistercrunch @kristw 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to