dannycjones commented on issue #3702: URL: https://github.com/apache/iceberg-rust/issues/3702#issuecomment-6957401375
What I'd really like to see is an incremental approach/proposal here. In particular: if we already have scans and writes in Rust, then what are the missing bits and do they all need to be implemented to benefit or only bits? What is the expected improvement of each bit? Compaction is one of those things where people implement it in different ways and the Spark-based compaction procedure is just one implementation. Are we wanting to replicate that same algorithm or do something different? Does implementing the same algorithm actually get us much improvement over what can be done with today's Comet plugin? This is a bit of a dump of my initial thoughts, I'd love to come back to this and dig a bit deeper. In terms of roadmap, I think answers to above will help formulate where that sits among the community. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
