aminghadersohi opened a new pull request, #45148: URL: https://github.com/apache/superset/pull/45148
### SUMMARY `get_catalog` set `truncated=true` whenever any item on the page had its name or description shortened, or an oversized URL omitted. That also happened on a complete final page with `next_cursor=null`. A consumer that reads `truncated` as "entries were held back" (the documented meaning of the byte-bound case) then sees a page that claims to be incomplete but gives no way to continue. With a larger `page_size`, every asset after a shortened one lands on the same final page, so the whole listing looks incomplete. Example: 20 visible datasets, one with a description over 500 characters, `page_size=50` → all 20 items, `next_cursor=null`, `truncated=true`. Changes: - `truncated` now means only that the page was cut to fit the response size bound. That case always returns a `next_cursor` that resumes right after the last returned item, so a page without a cursor is never flagged truncated. - New `fields_shortened` flag reports per-item text shortening and omitted URLs, computed for the items actually on the page. It does not mean entries are missing. - Bounds are unchanged (100 items, 32 KiB, per-field caps). The response only gains a field; cursors and request shape are unchanged. - Tool docstring and schema field descriptions spell out the meaning of `next_cursor`, `truncated` and `fields_shortened`. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF N/A ### TESTING INSTRUCTIONS ``` pytest tests/unit_tests/mcp_service/catalog ``` New tests walk every page through `next_cursor` and check that a page without a cursor is never truncated, a truncated page always has a cursor, and pages neither overlap nor skip rows. They cover: - the reported case: complete page with a shortened description - `fields_shortened` follows the items on each page - all four asset types with `page_size` 1, 2, 50 and 100 (fewer than, equal to and more than the row count) - 99/100/101/250 rows at `page_size=100` - byte-bound pages: the next page starts at the row right after the last one returned - search together with a cursor - rows hidden by permissions do not create a cursor or a truncated flag The existing low `SQLALCHEMY_DAO_MAX_PAGE_SIZE` test still covers a `page_size` above the DAO cap. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
