zhengruifeng opened a new pull request, #58692: URL: https://github.com/apache/spark/pull/58692
### What changes were proposed in this pull request? This PR adds an internal `VectorAffineTransform` Catalyst expression, registered as `ml_vector_affine_transform`, that computes `vector(i) * scale(i) + shift(i)` directly on the SQL struct representation of MLlib vectors. The expression supports interpreted and code-generated evaluation for dense and sparse vectors. It preserves sparse output when the input vector is sparse and the shift is zero; a nonzero shift produces a dense vector. This PR only adds the expression and does not use it in ML implementations. ### Why are the changes needed? Several ML feature transformations perform per-feature scaling and shifting through Scala UDFs. This expression provides a reusable Catalyst building block for future optimizations without deserializing vectors into MLlib objects. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? New tests cover interpreted and code-generated evaluation with dense and sparse vectors, sparse output preservation, nonzero shifts, null and empty vectors, infinite and NaN values, and mismatched dimensions. ```text build/sbt 'catalyst/testOnly *VectorAffineTransformSuite' ./dev/lint-scala ``` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
