Ruifeng Zheng created SPARK-61228:
-------------------------------------
Summary: Add a dedicated vector_to_array Catalyst expression
Key: SPARK-61228
URL: https://issues.apache.org/jira/browse/SPARK-61228
Project: Spark
Issue Type: Improvement
Components: ML
Affects Versions: 5.0.0
Reporter: Ruifeng Zheng
The current vector_to_array implementation uses Scala UDFs that deserialize
vector objects. Add a dedicated Catalyst ML expression that accepts an input
expression and a target datatype (double or float), and accepts both a vector
SQL struct and any UDT whose underlying SQL type matches the vector schema.
Use the expression in InternalFunctionRegistration and register
ml_vector_to_array in FunctionRegistry. Evaluate directly on Catalyst rows and
arrays in interpreted and generated execution, with coverage for dense, sparse,
empty, and matching-UDT inputs.
Pull request: https://github.com/apache/spark/pull/60224
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
