zhengruifeng opened a new pull request, #58240: URL: https://github.com/apache/spark/pull/58240
### What changes were proposed in this pull request? This PR updates `MLUtils` vector and matrix column conversion helpers to use `unwrap_udt` + `wrap_udt` instead of row-by-row UDF conversion. It applies to: - `convertVectorColumnsToML` - `convertVectorColumnsFromML` - `convertMatrixColumnsToML` - `convertMatrixColumnsFromML` The existing column selection, type validation, and metadata preservation behavior is unchanged. ### Why are the changes needed? The old and new MLlib vector/matrix UDTs share the same SQL backing schema. After SPARK-58875, Spark can re-wrap the same underlying SQL value with the target UDT directly, avoiding the serialization overhead from converting each value through a Scala UDF. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added nullable vector and matrix column conversion coverage in `MLUtilsSuite`. Ran: ``` build/sbt 'mllib/testOnly org.apache.spark.mllib.util.MLUtilsSuite -- -z "convertVectorColumnsToML" -z "convertVectorColumnsFromML" -z "convertMatrixColumnsToML" -z "convertMatrixColumnsFromML" -z "convert nullable vector columns" -z "convert nullable matrix columns"' ``` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
