[
https://issues.apache.org/jira/browse/IGNITE-27516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr updated IGNITE-27516:
-------------------------------
Description:
Implement iterative (streaming) cursor reading for SQL query results in CLI to
avoid loading entire result sets into memory.
_Problem:_
Currently, the CLI loads the entire query result into memory before displaying.
For large result sets (millions of rows), this causes:
* Out of memory errors
* Long delays before any output appears
* Poor user experience for large queries
_Solution:_
Read cursor data iteratively, displaying results in batches as they arrive.
_New config keys:_
* ignite.cli.sql.default-limit (integer, default: unlimited or 1000)
* ignite.cli.sql.batch-size (integer, default: 100)
_Command-line flags:_
* --limit N - limit number of rows returned
_Requirements:_
* Memory usage should be bounded regardless of result set size
* First rows should appear quickly (low time-to-first-row)
* Must integrate with pager (IGNITE-27514) - pager receives data incrementally
* Show "... N more rows" indicator when limit applied
was:
Implement iterative (streaming) cursor reading for SQL query results in CLI to
avoid loading entire result sets into memory.
_Problem:_
Currently, the CLI loads the entire query result into memory before displaying.
For large result sets (millions of rows), this causes:
* Out of memory errors
* Long delays before any output appears
* Poor user experience for large queries
_Solution:_
Read cursor data iteratively, displaying results in batches as they arrive.
_Implementation approach:_
* Modify SQL execution to use streaming/cursor-based result fetching
* Display rows as they are fetched (batch by batch)
* When combined with pager: pipe batches through pager incrementally
* Add --limit N flag to cap result rows
* Show progress indicator: "Displaying row X of Y..." or streaming indicator
_New config keys:_
* ignite.cli.sql.default-limit (integer, default: unlimited or 1000)
* ignite.cli.sql.batch-size (integer, default: 100)
_Command-line flags:_
* --limit N - limit number of rows returned
_Key files to investigate:_
* SqlQueryCall.java - SQL execution
* SqlQueryResultDecorator.java - result formatting
* JdbcQueryCursorImpl.java - cursor implementation
_Requirements:_
* Memory usage should be bounded regardless of result set size
* First rows should appear quickly (low time-to-first-row)
* Must integrate with pager (Option 1) - pager receives data incrementally
* Show "... N more rows" indicator when limit applied
> CLI: Implement iterative cursor reading for SQL results
> -------------------------------------------------------
>
> Key: IGNITE-27516
> URL: https://issues.apache.org/jira/browse/IGNITE-27516
> Project: Ignite
> Issue Type: Improvement
> Reporter: Aleksandr
> Priority: Major
> Labels: ignite-3
>
> Implement iterative (streaming) cursor reading for SQL query results in CLI
> to avoid loading entire result sets into memory.
> _Problem:_
> Currently, the CLI loads the entire query result into memory before
> displaying. For large result sets (millions of rows), this causes:
> * Out of memory errors
> * Long delays before any output appears
> * Poor user experience for large queries
> _Solution:_
> Read cursor data iteratively, displaying results in batches as they arrive.
> _New config keys:_
> * ignite.cli.sql.default-limit (integer, default: unlimited or 1000)
> * ignite.cli.sql.batch-size (integer, default: 100)
> _Command-line flags:_
> * --limit N - limit number of rows returned
> _Requirements:_
> * Memory usage should be bounded regardless of result set size
> * First rows should appear quickly (low time-to-first-row)
> * Must integrate with pager (IGNITE-27514) - pager receives data incrementally
> * Show "... N more rows" indicator when limit applied
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
