schenksj commented on issue #174:
URL:
https://github.com/apache/datafusion-comet/issues/174#issuecomment-4233212033
Hi! I'm Scott Schenkein, author of https://indextables.io. While building
IndexTables I spent significant time working with both Comet and
delta-kernel-rs, and wanted to give back to the project.
I've put together a PR that adds native Delta Lake read support to Comet
via delta-kernel-rs: https://github.com/apache/datafusion-comet/pull/3932
It matches the existing Iceberg native scan path feature-for-feature —
deletion vectors, column mapping, DPP, split-mode serialization, predicate
pushdown, and cloud storage support are all covered.
The design uses kernel for log replay on the driver and routes data reads
through Comet's tuned ParquetSource.
Would appreciate any review feedback from the community. Happy to iterate
on the design or address concerns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
