lidavidm commented on PR #4694: URL: https://github.com/apache/arrow-adbc/pull/4694#issuecomment-6728728150
> Since filters are already statement-scoped, what about supplying them via `AdbcStatementBind` and exposing them via `AdbcStatementGetParameterSchema` instead of options? That gives typed values (list for the *_types filters instead of comma-separated strings), reuses existing discovery instead of collection_filters/filter_type, makes "clear filters" just a rebind, and multi-row binds allow batched lookups (e.g., columns for 50 tables in one call). It would need by-name parameter matching (omitted field = no filter, to distinguish from filtering on NULL) and a convention for required filters, likely field metadata. Hmm, I guess my concern is that it's relatively expensive/annoying to deal with Arrow values for a single row. But I'll give this some more thought... > It had also occurred to me that `ExecutePartitions` (possibly incremental) could cover parallel and lazy reads but not resuming across separate fetches, so I think we'd still want a token for pagination. It could be a bindable parameter, with the next token returned in the result schema metadata. That's an interesting idea. As for pagination, I'm mostly concerned about making sure there's a way to expose it, and we can work out the details with a candidate implementation > Another idea: we could require `ExecuteSchema` support when this feature is enabled and use that instead of returning a JSON-encoded schema from the `meta` call. This shrinks the dependence on `arrow.schema_json` to the per-row schema fields in `table_schema`, `routine_parameter_schema` and `routine_result_schema` and would unblock progress on this feature for any metadata calls that do _not_ need to return individual schemas (as `meta` is the most-blocking thing here). How would this work exactly, a schema where each field is a struct type corresponding to one of the collections? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
