robdiciuccio opened a new pull request #9096: SQL Lab: Use numpy structured arrays, fallback to JSON serialization URL: https://github.com/apache/incubator-superset/pull/9096 ### CATEGORY Choose one - [x] Bug Fix - [ ] Enhancement (new features, refinement) - [ ] Refactor - [ ] Add tests - [ ] Build / Development Environment - [ ] Documentation ### SUMMARY Some SQL Lab queries were failing due to deeply nested data structures in columns. Additionally, `numpy` was sometimes getting confused when only one row was returned containing a single column of nested data. This PR 1) leverages `numpy` [structured arrays](https://docs.scipy.org/doc/numpy/user/basics.rec.html) to improve columnar data access, and 2) attempts JSON serialization of nested data structures that PyArrow can't handle natively. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF ### TEST PLAN Test against various database backends containing columns with nested data. ### ADDITIONAL INFORMATION <!--- Check any relevant boxes with "x" --> <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue --> - [ ] Has associated issue: - [ ] Changes UI - [ ] Requires DB Migration. - [ ] Confirm DB Migration upgrade and downgrade tested. - [ ] Introduces new feature or API - [ ] Removes existing feature or API ### REVIEWERS @john-bodley @graceguo-supercat @michellethomas @villebro @dpgaspar @craig-rueda @mistercrunch
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
