mistercrunch commented on a change in pull request #7641: [SQL Lab] Clarify SQL
Lab query and display limits
URL:
https://github.com/apache/incubator-superset/pull/7641#discussion_r290131709
##########
File path: superset/views/core.py
##########
@@ -2710,8 +2710,10 @@ def sql_json(self):
rendered_query,
return_results=True,
user_name=g.user.username if g.user else None)
+
+
payload = json.dumps(
- data,
+ apply_display_max_row_limit(data),
Review comment:
In the case of a sync request, we may want to apply `DISPLAY_MAX_ROW`
straight onto the SQL itself.
The reason we have 2 limits in the first place is because we want to store
the 1M rows in the results backend (say S3) to allow for CSV download of the 1M
row as a web request that reads from the s3 bucket, yet not crash the web
browser.
Here's another related idea that's been discussed before around this to get
data results faster to the user. It's the idea of doing a first dump after a
`cursor.fetch_many(10000)`, we'd probably dump that so a specific cache key,
and introduce a new query state: `SAMPLES_DELIVERED` (or something like that)
and then fetch the rest `cursor.fetch_all()` before reaching `SUCCESS`, and
dump the whole result set into another cache key. I think ultimately that's
where we want to go.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
