aminghadersohi opened a new pull request, #41698:
URL: https://github.com/apache/superset/pull/41698
### SUMMARY
`get_dashboard_info` (and the other MCP "info" tools: `get_chart_info`,
`get_dataset_info`, `get_instance_info`) silently truncates oversized list
fields like `charts` and `native_filters` down to a hardcoded 30 items whenever
the response exceeds the configured token budget, regardless of how much
headroom is actually available. A dashboard with 40 charts is truncated just as
harshly as one with 4,630 charts.
Root cause: `ResponseSizeGuardMiddleware` protects LLM clients from
oversized tool responses by calling `truncate_oversized_response()`, whose
Phase 2 (`_truncate_lists`) applied a non-configurable module constant
`_MAX_LIST_ITEMS = 30`.
This PR:
- Raises the default list-item cap from 30 to 100 (matching the existing
`MAX_PAGE_SIZE` constant used elsewhere for pagination), via a new
`DEFAULT_MAX_LIST_ITEMS` constant.
- Threads the cap through as a configurable parameter:
`truncate_oversized_response(..., max_list_items=...)` →
`ResponseSizeGuardMiddleware(max_list_items=...)` →
`MCP_RESPONSE_SIZE_CONFIG["max_list_items"]`, mirroring the existing
`token_limit`/`warn_threshold_pct` config pattern.
- Leaves Phase 4's aggressive `max_items=10` last-resort backstop untouched.
- Documents a real, unbounded escape hatch for dashboards that exceed even a
raised cap (e.g. Medialab's 463 charts): `list_charts` already supports
filtering to a single dashboard (`filters=[{"col": "dashboards", "opr": "eq",
"value": <dashboard id>}]`) with genuine `page`/`page_size` pagination, so an
agent can retrieve the complete chart list regardless of size. This is now
surfaced in `get_dashboard_info`'s docstring and in the
`DashboardInfo.charts`/`native_filters` field descriptions, along with
`chart_count` as the signal for detecting truncation (compare `len(charts)` to
`chart_count`).
Operators with tenants that have unusually large dashboards can also raise
`max_list_items` further via `MCP_RESPONSE_SIZE_CONFIG` in `superset_config.py`.
**What this does not do:** true server-side pagination of
`get_dashboard_info` itself (e.g. `charts_page`/`charts_page_size` params baked
into the tool). That would be a larger change; the `list_charts`
filter-by-dashboard path already provides a complete, uncapped way to enumerate
a dashboard's charts, so this PR wires up and documents that existing
capability rather than building a parallel pagination mechanism.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A — backend-only change.
### TESTING INSTRUCTIONS
- `pytest tests/unit_tests/mcp_service/utils/test_token_utils.py
tests/unit_tests/mcp_service/test_middleware.py
tests/unit_tests/mcp_service/dashboard/`
- New regression tests simulate the reported large-dashboard payload (463
charts, 48 native_filters):
- With the new default cap (100), `native_filters` (48 items) is left
untouched while `charts` (463 items) is truncated to 100.
- A custom `max_list_items` value truncates both fields to that custom cap.
- New tests assert the `list_charts` pagination escape hatch stays
documented in `get_dashboard_info`'s docstring and in `DashboardInfo`'s field
descriptions.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
