paleolimbot commented on issue #7063: URL: https://github.com/apache/arrow-rs/issues/7063#issuecomment-2790967285
In https://github.com/apache/arrow-rs/pull/7398 and https://github.com/apache/datafusion/pull/15663 I mostly found that I was adding an additional "this type isn't supported" to the end of enums with one or more entires that already had other types that weren't supported. The tradeoff is I think that arrow-rs would take on the responsibility of ensuring the hard-to-implement operations (filter, take, concatenate) propagate the type so that extension authors/frameworks built on arrow don't have to (there's a few PRs in DataFusion that demonstrate that updating the DataType representation has quite large footprint). I would argue that by excluding the option in arrow-rs, we aren't necessarily eliminating any complexity, we're just punting it on to other projects. I've tried building the PR implementing this against DataFusion to check the impact on downstream code ( https://github.com/apache/datafusion/pull/15663 )...are there other projects with a high Arrow surface area that might have a larger disruption? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
