andygrove commented on code in PR #4763: URL: https://github.com/apache/datafusion-comet/pull/4763#discussion_r3538627025
########## native/spark-expr/src/conversion_funcs/cast.rs: ########## Review Comment: Fixed in bdf9491. You are right that this arm was reachable and panicking: with `spark.sql.binaryOutputStyle=UTF8`, `ToPrettyString`/`show()` over a binary column with non-UTF-8 bytes lands here, and `String::from_utf8(..).unwrap()` panics the executor. Spark renders it via `UTF8String.fromBytes(bytes).toString`, i.e. `new String(bytes, UTF_8)`, so this now routes through the shared `decode_utf8_spark_lossy` and yields the same `U+FFFD` semantics as the plain cast. Added a `cast_binary_to_string` test with `binary_output_style = Utf8` over the same invalid-byte inputs (including the surrogate-range `[ED A0 80]` collapse) to lock the behavior in. ########## native/spark-expr/src/conversion_funcs/cast.rs: ########## Review Comment: test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
