[
https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786924#comment-16786924
]
Anne Holler commented on SPARK-26247:
-------------------------------------
Hi, Sean, Thanks for your comments!
>There are two issues here – load time of the model, and scoring outside Spark?
Yes.
>What's the issue with load time? surely that happens once before serving.
>Is it that you want to read a PipelineModel directly, without Spark? OK,
>that's more interesting.
Yes, load only happens once before serving, but having the load time be
exceptionally slow
(originally 8x-44x slower than our custom pb representation) impacts the
velocity with which
new online nodes serving a set of pipeline models can be brought up and become
ready to serve.
Yes, we read a PipelineModel directly, without Spark, without changing the
on-disk Spark Pipeline representation.
>The thing can't be scored without Spark without some form of transformation
>and that's mostly what MLeap does.
>I'm trying to understand why this is enough different that it needs to be in
>Spark as the 'blessed' solution.
>Putting the maintenance onto this project is more bug than feature. There are
>unfortunately several incomplete
>attempts to do something like this: mllib-local, PMML export. I hesitate to
>add another. If the idea is
>supporting single-instance scoring of models, that partly exists in some
>models already and in mllib-local.
MLeap, PMML export, PFA, solve the problem by creating a different model
representation and an external
reimplementation of the scoring, which increases the risk of inconsistent
serving and adds latency and toil
to supporting Spark Transformer updates. Adding single-instance (also
small-list) scoring of models natively
in Spark leverages when practical the low-level Spark methods used by the
transform path, enhancing Spark
itself for online serving capability at what we have found to be minimal cost,
based on our having used this
approach in production across Spark upgrades for several years.
Let me provide a POC patch, to give a direct sense of the changes. I can move
the changes from Spark 2.3.2
(which is our current production release) to master and go through the
company's external code publish approval
process, both of which hopefully can be done quickly. Thanks again for your
comments.
> SPIP - ML Model Extension for no-Spark MLLib Online Serving
> -----------------------------------------------------------
>
> Key: SPARK-26247
> URL: https://issues.apache.org/jira/browse/SPARK-26247
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 2.1.0
> Reporter: Anne Holler
> Priority: Major
> Labels: SPIP
> Attachments: SPIPMlModelExtensionForOnlineServing.pdf
>
>
> This ticket tracks an SPIP to improve model load time and model serving
> interfaces for online serving of Spark MLlib models. The SPIP is here
> [https://docs.google.com/a/uber.com/document/d/e/2PACX-1vRttVNNMBt4pBU2oBWKoiK3-7PW6RDwvHNgSMqO67ilxTX_WUStJ2ysUdAk5Im08eyHvlpcfq1g-DLF/pub]
>
> The improvement opportunity exists in all versions of spark. We developed
> our set of changes wrt version 2.1.0 and can port them forward to other
> versions (e.g., we have ported them forward to 2.3.2).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
