vustef commented on issue #1636: URL: https://github.com/apache/iceberg-rust/issues/1636#issuecomment-3249152053
I realize this may not be possible to support by the iceberg-rust alone, without some query processing engine. Examples that I gave above use Spark to iterate through rows (and for example deduplicate them if `net_changes` is enabled). However, it'd be useful if we could open up extension points so that a query processing engine together with this library can achieve what Spark does with the Java library. For example, it'd be useful to make it possible to retrieve only data that was added or removed in between some snapshots. One could modify `plan_files` slightly to allow filtering manifests whose `added_snapshot_id` is not in the desired snapshot IDs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
