johnsolomonj opened a new issue, #16707:
URL: https://github.com/apache/pinot/issues/16707
### Summary
Add native cursor-based pagination support to the Java client
(`pinot-clients/pinot-java-client`) to leverage the server-side ResponseStore
APIs introduced in Apache Pinot 1.3.0.
### Motivation
Large result sets require efficient pagination. While Pinot’s brokers
already implement cursor pagination via the ResponseStore (POST /getCursor, GET
/{requestId}/results, DELETE /{requestId}) APIs, the Java client does not
expose them directly. Adding this feature will:
- Enable Java applications to page through results without loading all rows
into memory
- Provide seek‐forward and seek‐backward semantics for analytics
- Reduce client‐side complexity by encapsulating cursor management
### Proposal
#### API Design
Extend `Connection` (or create `CursorConnection`) with methods:
`CursorResultSetGroup executeWithCursor(String sql, int numRows);`
`CursorResultSetGroup fetchNextPage(String requestId, int offset, int
numRows);`
`CursorMetadata getCursorMetadata(String requestId);`
`void closeCursor(String requestId);`
#### Transport Enhancements
Update `JsonAsyncHttpPinotClientTransport` (and gRPC transport) to call
ResponseStore endpoints:
`POST` /query/sql?getCursor=true&numRows=<n>
`GET` /responseStore/{requestId}/results?offset=<o>&numRows=<n>
`GET` /responseStore/{requestId}
`DELETE` /responseStore/{requestId}
#### Result Handling
- Implement `CursorResultSetGroup` extending `ResultSetGroup` to track
`requestId`, pagination metadata, and provide iterator methods.
- Create `CursorMetadata` for expiration and broker info.
#### Testing
- Unit tests for each new API method, error conditions, and cleanup.
- Integration tests against a Dockerized Pinot cluster to validate full
pagination lifecycle.
#### Documentation
- Add JavaDoc examples for cursor methods.
- Update client user guide with cursor pagination section and sample code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
