[
https://issues.apache.org/jira/browse/SPARK-61228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruifeng Zheng updated SPARK-61228:
----------------------------------
Component/s: SQL
Summary: [SQL][ML] Add a dedicated vector_to_array Catalyst expression
(was: Add a dedicated vector_to_array Catalyst expression)
> [SQL][ML] Add a dedicated vector_to_array Catalyst expression
> -------------------------------------------------------------
>
> Key: SPARK-61228
> URL: https://issues.apache.org/jira/browse/SPARK-61228
> Project: Spark
> Issue Type: Improvement
> Components: ML, SQL
> Affects Versions: 5.0.0
> Reporter: Ruifeng Zheng
> Priority: Major
> Labels: pull-request-available
>
> The current vector_to_array implementation uses Scala UDFs that deserialize
> vector objects. Add a dedicated Catalyst ML expression that accepts an input
> expression and a target datatype (double or float), and accepts both a vector
> SQL struct and any UDT whose underlying SQL type matches the vector schema.
> Use the expression in InternalFunctionRegistration and register
> ml_vector_to_array in FunctionRegistry. Evaluate directly on Catalyst rows
> and arrays in interpreted and generated execution, with coverage for dense,
> sparse, empty, and matching-UDT inputs.
> Pull request: https://github.com/apache/spark/pull/60224
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
