gengliangwang commented on PR #59424: URL: https://github.com/apache/spark/pull/59424#issuecomment-6199181641
I opened a companion Rust SDK prototype in #59520 for this PR's C ABI, pinned to `51af65c`. It separates driver-side `BatchSource` / `ScanBuilder` from a serializable `BatchReaderFactory`, caches immutable plans after pushdown negotiation, and exposes a lifetime-bound task context with cooperative cancellation. It advertises only `SPARK_DS_BATCH_READ`, leaving unsupported function slots null, without changing the native ABI. The SDK handles the function table, plan serialization, provider errors and unwinding panics, and Arrow ownership. It includes a Range example, packaging instructions, and passing standalone Rust/C tests. End-to-end Spark validation is still pending. Feedback on the connector API and lifecycle is welcome! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
