bito-code-review[bot] commented on code in PR #44604:
URL: https://github.com/apache/superset/pull/44604#discussion_r4139859172
##########
superset/db_engine_specs/bigquery.py:
##########
@@ -465,13 +465,17 @@ def fetch_data(cls, cursor: Any, limit: int | None =
None) -> list[tuple[Any, ..
# First batch already covers the budget or the result set
if rows_fetched < initial_batch_size or remaining_rows <= 0:
+ # A full sample can also be the entire result. Confirm an
omitted
+ # row; a bounded cursor leaves its own EOF probe to the
executor.
memory_limited = (
- remaining_rows <= 0 and rows_fetched == initial_batch_size
+ remaining_rows <= 0
+ and rows_fetched == initial_batch_size
+ and bool(cursor.fetchmany(1))
Review Comment:
<!-- Bito Reply -->
The suggestion is valid and addresses a potential data loss issue. When the
first batch of rows exactly matches the budget, the EOF probe
`cursor.fetchmany(1)` consumes an additional row that is currently discarded
because the function returns `first_batch` directly. Capturing this probe
result and appending it to `first_batch` when non-empty ensures that the
truncation status is correctly identified and no rows are silently dropped.
**superset/db_engine_specs/bigquery.py**
```
# First batch already covers the budget or the result set
if rows_fetched < initial_batch_size or remaining_rows <= 0:
# A full sample can also be the entire result. Confirm an
omitted
# row; a bounded cursor leaves its own EOF probe to the
executor.
memory_limited = (
remaining_rows <= 0
and rows_fetched == initial_batch_size
and bool(cursor.fetchmany(1))
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
