andy-clapson opened a new issue, #37612:
URL: https://github.com/apache/superset/issues/37612
### Bug description
### Bug description
There are two related bugs that combine to make SQL Lab's "Data preview" tab
spin forever on large tables:
#### Bug 1: `queryLimit` not set on preview queries
In `superset-frontend/src/SqlLab/actions/sqlLab.ts`, the
`runTablePreviewQuery` function builds a `dataPreviewQuery` object (line ~1332)
but **never sets `queryLimit`**:
```typescript
const dataPreviewQuery: Query = {
id: newTable.previewQueryId ?? nanoid(11),
dbId: dbId as number | undefined,
// ... other fields ...
isDataPreview: true,
// queryLimit is MISSING
};
```
The `PREVIEW_QUERY_LIMIT = 100` constant (line 66 of
`TablePreview/index.tsx`) is only used for the `displayLimit` and
`defaultQueryLimit` UI props — it is never sent to the backend.
When `runQuery` sends the payload, `queryLimit` is `undefined`. On the
backend, `_get_limit_param` in `superset/sqllab/sqllab_execution_context.py`
evaluates:
```python
limit = apply_max_row_limit(query_params.get("queryLimit") or 0)
```
`None or 0` → `0`, and `apply_max_row_limit(0)` returns `SQL_MAX_ROW`
(default **100,000**). Then `apply_limit` adds +1 for the overflow detection
row, so the actual database query runs with `LIMIT 100001` instead of `LIMIT
101`.
This means the "100 row preview" actually fetches up to **100,001 rows**
from the database, which on large tables produces massive payloads that can
overwhelm the results backend.
**Fix:** Add `queryLimit: PREVIEW_QUERY_LIMIT` (or just `100`) to the
`dataPreviewQuery` object in `runTablePreviewQuery`.
#### Bug 2: `float` vs `decimal.Decimal` TypeError in results error handler
In `superset/commands/sql_lab/results.py`, line ~96:
```python
query_age_seconds = now_as_float() - (
self._query.end_time if self._query.end_time else now_as_float()
)
```
- `now_as_float()` returns a Python `float`
- `self._query.end_time` is a `Column(Numeric(precision=20, scale=6))` which
SQLAlchemy returns as `decimal.Decimal`
This raises `TypeError: unsupported operand type(s) for -: 'float' and
'decimal.Decimal'`, which crashes the error handler into a 500 instead of
returning the intended 410 "re-run your query" message. The frontend receives
no actionable error, so the spinner runs indefinitely.
**Fix:** Cast to float: `float(self._query.end_time) if self._query.end_time
else now_as_float()`
### How to reproduce
1. Connect a database with a large table in SQL Lab
2. Expand the table in the schema explorer
3. Click the "Data preview" tab
4. Observe the spinner never resolves
Check the Superset logs for:
```
TypeError: unsupported operand type(s) for -: 'float' and 'decimal.Decimal'
```
### Expected results
Data preview should fetch 100 rows quickly and display them, or show a clear
error message if results are unavailable.
### Actual results
The preview fetches up to 100,001 rows from the database. If the results
backend fails to store or return the large payload, the error handler crashes
with a TypeError, and the frontend shows an infinite spinner with no error
message.
### Environment
- Apache Superset (current master)
- Results backend: Redis (via `cachelib.RedisCache`)
- Database: PostgreSQL
### Checklist
- [x] I have searched Superset for duplicate issues
- [x] I have provided steps to reproduce
### Screenshots/recordings
_No response_
### Superset version
master / latest-dev
### Python version
3.9
### Node version
16
### Browser
Chrome
### Additional context
_No response_
### Checklist
- [ ] I have searched Superset docs and Slack and didn't find a solution to
my problem.
- [ ] I have searched the GitHub issue tracker and didn't find a similar bug
report.
- [ ] I have checked Superset's logs for errors and if I found a relevant
Python stacktrace, I included it here as text in the "additional context"
section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
