wirybeaver opened a new issue, #118: URL: https://github.com/apache/datafusion-java/issues/118
## Problem Java-implemented `TableProvider`s currently expose only a full-schema `scan(BufferAllocator)`. The Rust adapter receives a projection and limit, but does not pass them to Java; Java sources also cannot report DataFusion's Exact/Inexact/Unsupported filter-pushdown capabilities. This prevents external Java sources from selecting a selective access path while preserving DataFusion's optimizer semantics. ## Proposed scope Add an opt-in scan-planning contract alongside the existing API: - Immutable scan requests with projection indices (including an empty projection), structured DataFusion expression protos and an optional limit. - Per-expression Exact/Inexact/Unsupported capability reporting. - A scan plan that returns a fresh allocator-compatible Arrow reader only during physical execution. Planning must not read source data. - Source projection with schema validation, while existing providers retain full-schema scans and the adapter's projection fallback. The implementation would remain a thin, source-agnostic Java/JNI adapter. No storage SDK, source-specific predicate rules, query-language semantics or DataFusion-core changes are proposed. Tests would use an in-memory selective source to cover exact and inexact filtering, residual filters with LIMIT, projection, independent scans and resource/error handling. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
