andygrove commented on PR #27067: URL: https://github.com/apache/datafusion/pull/27067#issuecomment-7018046701
> Is it really worth handling the nested case when the non-nested case has been broken for months (years?)? This is a regression since DF 55 and we had to ignore a test in Comet as a workaround. This is a valid backport candidate according to the release management guide. Not fixing it leaves a release build able to hand out a StringArray with invalid UTF-8, which is undefined behavior for any safe code that reads it as &str. On 55.2.0 the same query returned an error. I agree that it is not ideal that this issue already exists for top level columns. We can at least stop this unsound behavior from spreading farther? > This also disables functionality (things currently work okay for valid UTF-8, which I'd think is the majority of the use case of this non-default option). No, I don't think it does? I added a test to confirm. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
