Re: Serving Spark ML models via a regular Python web app

Hi Chris, I was just checking out your project. I mentioned we use MLeap to serve predictions from a trained Spark ML RandomForest model. How would I do that with pipeline.io ? It isn't clear to me. Thanks! Michael > On Aug 11, 2016, at 9:42 AM, Chris Fregly wrote: > >

Re: Serving Spark ML models via a regular Python web app

Thanks for the additional reference Chris. Sounds like there are a few independent projects addressing this story. On Thu, Aug 11, 2016 at 12:42 PM Chris Fregly wrote: > And here's a recent slide deck on the pipeline.io that summarizes what > we're working on (all open source): > > > https://www

Re: Serving Spark ML models via a regular Python web app

And here's a recent slide deck on the pipeline.io that summarizes what we're working on (all open source): https://www.slideshare.net/mobile/cfregly/advanced-spark-and-tensorflow-meetup-08042016-one-click-spark-ml-pipeline-deploy-to-production mleap is heading the wrong direction and reinventi

Re: Serving Spark ML models via a regular Python web app

this is exactly what my http://pipeline.io project is addressing. check it out and send me feedback or create issues at that github location. > On Aug 11, 2016, at 7:42 AM, Nicholas Chammas > wrote: > > Thanks Michael for the reference, and thanks Nick for the comprehensive > overview of exi

Re: Serving Spark ML models via a regular Python web app

Thanks Michael for the reference, and thanks Nick for the comprehensive overview of existing JIRA discussions about this. I've added myself as a watcher on the various tasks. On Thu, Aug 11, 2016 at 3:02 AM Nick Pentreath wrote: > Currently there is no direct way in Spark to serve models without

Re: Serving Spark ML models via a regular Python web app

Currently there is no direct way in Spark to serve models without bringing in all of Spark as a dependency. For Spark ML, there is actually no way to do it independently of DataFrames either (which for single-instance prediction makes things sub-optimal). That is covered here: https://issues.apach

Re: Serving Spark ML models via a regular Python web app

Nick, Check out MLeap: https://github.com/TrueCar/mleap . It's not python, but we use it in production to serve a random forest model trained by a Spark ML pipeline. Thanks, Michael > On Aug 10, 2016, at 7:50 PM, Nicholas Chammas > wrote: > > Are there any

Serving Spark ML models via a regular Python web app

Are there any existing JIRAs covering the possibility of serving up Spark ML models via, for example, a regular Python web app? The story goes like this: You train your model with Spark on several TB of data, and now you want to use it in a prediction service that you’re building, say with Flask <