[
https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782876#comment-16782876
]
Sean Owen commented on SPARK-26247:
-----------------------------------
There are two issues here -- load time of the model, and scoring outside Spark?
What's the issue with load time? surely that happens once before serving. Is it
that you want to read a PipelineModel directly, without Spark? OK, that's more
interesting. The thing can't be scored without Spark without some form of
transformation and that's mostly what MLeap does. I'm trying to understand why
this is enough different that it needs to be in Spark as the 'blessed'
solution. Putting the maintenance onto this project is more bug than feature.
There are unfortunately several incomplete attempts to do something like this:
mllib-local, PMML export. I hesitate to add another.
If the idea is supporting single-instance scoring of models, that partly exists
in some models already and in mllib-local.
> SPIP - ML Model Extension for no-Spark MLLib Online Serving
> -----------------------------------------------------------
>
> Key: SPARK-26247
> URL: https://issues.apache.org/jira/browse/SPARK-26247
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 2.1.0
> Reporter: Anne Holler
> Priority: Major
> Labels: SPIP
> Attachments: SPIPMlModelExtensionForOnlineServing.pdf
>
>
> This ticket tracks an SPIP to improve model load time and model serving
> interfaces for online serving of Spark MLlib models. The SPIP is here
> [https://docs.google.com/a/uber.com/document/d/e/2PACX-1vRttVNNMBt4pBU2oBWKoiK3-7PW6RDwvHNgSMqO67ilxTX_WUStJ2ysUdAk5Im08eyHvlpcfq1g-DLF/pub]
>
> The improvement opportunity exists in all versions of spark. We developed
> our set of changes wrt version 2.1.0 and can port them forward to other
> versions (e.g., we have ported them forward to 2.3.2).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
