Ryan Avery created SPARK-46948:
----------------------------------
Summary: Support higher dimensional array return values in
predict_batch_udf beyond 1 or 2 dimensional arrays
Key: SPARK-46948
URL: https://issues.apache.org/jira/browse/SPARK-46948
Project: Spark
Issue Type: Improvement
Components: ML
Affects Versions: 3.5.0
Reporter: Ryan Avery
pyspark.ml.functions.predict_batch_udf does not support return types with more
than 2 dimensions:
[https://github.com/apache/spark/pull/37734#discussion_r1016156053]
Many computer vision models return ndarrays with 3 or 4 dimensions.
Segmentation returns 3 dimensions: [Category, H, W]and if there is a time
dimension, that's the fourth dimension.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
