lidavidm commented on issue #2709:
URL: https://github.com/apache/arrow-adbc/issues/2709#issuecomment-2817849554
Putting things together we need the following:
- Executing a SQL query can either give you partitions or (multiple) result
sets
- Maybe this can be unified to giving you a list of "promise" objects that
can be either immediately consumed to get a (partition of a) result set, or
serialized into a partition (if supported)
- This does lead to more API calls, though...
- Each result set needs to have a requested schema bound (so the driver can
convert if it's able to/if there are potentially multiple valid Arrow types for
a column)
- Result streams should use the async device interface, which handles
backpressure already
- Cancelling the statement will cancel the entire query?
- Dropping a substream shouldn't cancel the entire query?
- Do we need some way to potentially retry a stream, and ideally retrying a
stream from a given (row) offset?
- Some of these questions may be specific to Flight SQL, but I think it
generally applies to any sort of newfangled system with a REST or gRPC API
instead of an old-school database protocol
- We won't support seeking through streams (aka no actual cursors)
- We should standardize that readers should be safe to use if ADBC objects
are released (but they may error instead of producing data and/or ADBC objects
may error on release if there are unclosed streams)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
