tustvold commented on issue #5700: URL: https://github.com/apache/arrow-rs/issues/5700#issuecomment-2082243010
I believe the narrow crate has some similar goals and might be worth checking out. FWIW I likewise found the lack of strong typing annoying when I first encountered the crate but have come to appreciate it for a number of reasons: * API compatibility: we still struggle to preserve this, but it gets infinitely harder with generics exposing what are often implementation details * Type Erasure: especially when working on query engines, you very often don't know and don't want to have to know what the type of something is. Downcasting to the concrete type is obnoxious, verbose, and macro heavy * Compilation Time: by not exposing the generics we can ensure they get instantiated once, and use tricks to reduce the amount of code that gets generated. Moving away from generics significantly improved compilation times * Codegen: LLVM is a very finicky beast, and a lot of care has gone into making sure it properly vectorises code. This is very hard to do with generics that are instantiated in the call site My 2 cents is that arrow makes little sense in statically typed contexts, specialized code will almost always win out both for ergonomics and performance. Perhaps using crates like serde-arrow to convert back and forth where necessary -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
