No, ours is different. We required predictions within 10ms time frame so we
needed much less latency than that.

Every algorithm has some parameters. Correct? We took the parameters from
the mllib and used them to create ml package's model. ml package's model's
prediction time was much faster compared to mllib package's transformation.
So essentially use spark's distributed machine learning library to train
the model, save to S3, load from S3 in a different system and then convert
it into the vector based API model for actual predictions.

There were obviously some transformations involved but we didn't use
Pipeline for those transformations. Instead, we re-wrote them for the
Vector based API. I know it's not perfect but if we had used the
transformations within the pipeline that would make us dependent on spark's
distributed API and we didn't see how we will really reach our latency
requirements. Would have been much simpler and more DRY if the
PipelineModel had a predict method based on vectors and was not distributed.

As you can guess it is very much model-specific and more work. If we decide
to use another type of Model we will have to add conversion
code/transformation code for that also. Only if spark exposed a prediction
method which is as fast as the old machine learning package.

On Sat, Oct 15, 2016 at 8:42 PM, Nicolas Long <nicolasl...@gmail.com> wrote:

> Hi Sean and Aseem,
> thanks both. A simple thing which sped things up greatly was simply to
> load our sql (for one record effectively) directly and then convert to a
> dataframe, rather than using Spark to load it. Sounds stupid, but this took
> us from > 5 seconds to ~1 second on a very small instance.
> Aseem: can you explain your solution a bit more? I'm not sure I understand
> it. At the moment we load our models from S3 (
> RandomForestClassificationModel.load(..) ) and then store that in an
> object property so that it persists across requests - this is in Scala. Is
> this essentially what you mean?
> On 12 October 2016 at 10:52, Aseem Bansal <asmbans...@gmail.com> wrote:
>> Hi
>> Faced a similar issue. Our solution was to load the model, cache it after
>> converting it to a model from mllib and then use that instead of ml model.
>> On Tue, Oct 11, 2016 at 10:22 PM, Sean Owen <so...@cloudera.com> wrote:
>>> I don't believe it will ever scale to spin up a whole distributed job to
>>> serve one request. You can look possibly at the bits in mllib-local. You
>>> might do well to export as something like PMML either with Spark's export
>>> or JPMML and then load it into a web container and score it, without Spark
>>> (possibly also with JPMML, OpenScoring)
>>> On Tue, Oct 11, 2016, 17:53 Nicolas Long <nicolasl...@gmail.com> wrote:
>>>> Hi all,
>>>> so I have a model which has been stored in S3. And I have a Scala
>>>> webapp which for certain requests loads the model and transforms submitted
>>>> data against it.
>>>> I'm not sure how to run this quickly on a single instance though. At
>>>> the moment Spark is being bundled up with the web app in an uberjar (sbt
>>>> assembly).
>>>> But the process is quite slow. I'm aiming for responses < 1 sec so that
>>>> the webapp can respond quickly to requests. When I look the Spark UI I see:
>>>> Summary Metrics for 1 Completed Tasks
>>>> Metric    Min    25th percentile    Median    75th percentile    Max
>>>> Duration    94 ms    94 ms    94 ms    94 ms    94 ms
>>>> Scheduler Delay    0 ms    0 ms    0 ms    0 ms    0 ms
>>>> Task Deserialization Time    3 s    3 s    3 s    3 s    3 s
>>>> GC Time    2 s    2 s    2 s    2 s    2 s
>>>> Result Serialization Time    0 ms    0 ms    0 ms    0 ms    0 ms
>>>> Getting Result Time    0 ms    0 ms    0 ms    0 ms    0 ms
>>>> Peak Execution Memory    0.0 B    0.0 B    0.0 B    0.0 B    0.0 B
>>>> I don't really understand why deserialization and GC should take so
>>>> long when the models are already loaded. Is this evidence I am doing
>>>> something wrong? And where can I get a better understanding on how Spark
>>>> works under the hood here, and how best to do a standalone/bundled jar
>>>> deployment?
>>>> Thanks!
>>>> Nic

Reply via email to