Re: thought experiment: use spark ML to real time prediction

2015-11-27 Thread Nick Pentreath
her similar use case. > > Kind regards > > Andy > > > > > From: Nick Pentreath <nick.pentre...@gmail.com> > Date: Wednesday, November 18, 2015 at 4:03 AM > To: DB Tsai <dbt...@dbtsai.com> > Cc: "user @spark" <user@spark.apache.org> &g

Re: thought experiment: use spark ML to real time prediction

2015-11-27 Thread Nick Pentreath
;>> >>>>>> >>>>>> Sincerely, >>>>>> >>>>>> DB Tsai >>>>>> -------------- >>>>>> Web: https://www.dbtsai.com >>>>>> PGP Key ID: 0xAF08DF8

Re: thought experiment: use spark ML to real time prediction

2015-11-22 Thread Vincenzo Selvaggio
08DF8D >>>>> >>>>> On Thu, Nov 12, 2015 at 8:15 PM, Nirmal Fernando <nir...@wso2.com> >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Fri, Nov 13, 2015 at 2:04 AM, darren <dar...@ontr

Re: thought experiment: use spark ML to real time prediction

2015-11-22 Thread Andy Davidson
> >>>>> On Thu, Nov 12, 2015 at 8:15 PM, Nirmal Fernando <nir...@wso2.com> wrote: >>>>>> >>>>>> >>>>>> On Fri, Nov 13, 2015 at 2:04 AM, darren <dar...@ontrenet.com> wrote: >>>>>>> >>>&g

Re: thought experiment: use spark ML to real time prediction

2015-11-18 Thread Nick Pentreath
;>>> This is a very useful side effect of ML models. >>>>>> >>>>>> If mlib can't use models outside spark that's a real shame. >>>>>> >>>>> >>>>> Well you can as mentioned earlier. You don't need Spark

Re: thought experiment: use spark ML to real time prediction

2015-11-17 Thread Nick Pentreath
gt; >>>> Original message >>>> From: "Kothuvatiparambil, Viju" < >>>> viju.kothuvatiparam...@bankofamerica.com> >>>> Date: 11/12/2015 3:09 PM (GMT-05:00) >>>> To: DB Tsai <dbt...@dbtsai.com>, S

Re: thought experiment: use spark ML to real time prediction

2015-11-17 Thread DB Tsai
n as mentioned earlier. You don't need Spark runtime for >>>> predictions, save the serialized model and deserialize to use. (you need >>>> the Spark Jars in the classpath though) >>>> >>>>> >>>>> >>>>> Sent from my Verizo

RE: thought experiment: use spark ML to real time prediction

2015-11-12 Thread Kothuvatiparambil, Viju
; Adrian Tanase; user @spark; Xiangrui Meng; hol...@pigscanfly.ca Subject: Re: thought experiment: use spark ML to real time prediction I think the use-case can be quick different from PMML. By having a Spark platform independent ML jar, this can empower users to do the following, 1) PMML doesn't

RE: thought experiment: use spark ML to real time prediction

2015-11-12 Thread darren
Davidson <a...@santacruzintegration.com>, Adrian Tanase <atan...@adobe.com>, "user @spark" <user@spark.apache.org>, Xiangrui Meng <men...@gmail.com>, hol...@pigscanfly.ca Subject: RE: thought experiment: use spark ML to real time prediction I am glad to

Re: thought experiment: use spark ML to real time prediction

2015-11-12 Thread Felix Cheung
+1 on that. It would be useful to use the model outside of Spark. _ From: DB Tsai <dbt...@dbtsai.com> Sent: Wednesday, November 11, 2015 11:57 PM Subject: Re: thought experiment: use spark ML to real time prediction To: Nirmal Fernando <nir...@wso2.com&

Re: thought experiment: use spark ML to real time prediction

2015-11-12 Thread Nick Pentreath
ote: > >> +1 on that. It would be useful to use the model outside of Spark. >> >> >> _ >> From: DB Tsai <dbt...@dbtsai.com> >> Sent: Wednesday, November 11, 2015 11:57 PM >> Subject: Re: thought experiment: use spar

Re: thought experiment: use spark ML to real time prediction

2015-11-12 Thread DB Tsai
>, Sean Owen <so...@cloudera.com> >> Cc: Felix Cheung <felixcheun...@hotmail.com>, Nirmal Fernando < >> nir...@wso2.com>, Andy Davidson <a...@santacruzintegration.com>, Adrian >> Tanase <atan...@adobe.com>, "user @spark" <user@spark.apache.or

Re: thought experiment: use spark ML to real time prediction

2015-11-12 Thread Nirmal Fernando
rnando < > nir...@wso2.com>, Andy Davidson <a...@santacruzintegration.com>, Adrian > Tanase <atan...@adobe.com>, "user @spark" <user@spark.apache.org>, > Xiangrui Meng <men...@gmail.com>, hol...@pigscanfly.ca > Subject: RE: thought experiment: use

Re: thought experiment: use spark ML to real time prediction

2015-11-12 Thread DB Tsai
be useful to use the model outside of Spark. >> >> >> _ >> From: DB Tsai <dbt...@dbtsai.com> >> Sent: Wednesday, November 11, 2015 11:57 PM >> Subject: Re: thought experiment: use spark ML to real time prediction >> To: Nirmal Fer

Re: thought experiment: use spark ML to real time prediction

2015-11-11 Thread Nirmal Fernando
> > -adrian > > From: Andy Davidson > Date: Tuesday, November 10, 2015 at 11:31 PM > To: "user @spark" > Subject: thought experiment: use spark ML to real time prediction > > Lets say I have use spark ML to train a linear model. I know I can save > and load the mo

Re: thought experiment: use spark ML to real time prediction

2015-11-11 Thread Adrian Tanase
outside of spark – loading the precomputed model and calling .predict on it… -adrian From: Andy Davidson Date: Tuesday, November 10, 2015 at 11:31 PM To: "user @spark" Subject: thought experiment: use spark ML to real time prediction Lets say I have use spark ML to train a linear model

Re: thought experiment: use spark ML to real time prediction

2015-11-11 Thread DB Tsai
pplications/content/twitter_classifier/predict.html >> >> Maybe you can find a way to extract portions of MLLib and run them >> outside of spark – loading the precomputed model and calling .predict on it… >> >> -adrian >> >> From: Andy Davidson >> Date: Tuesd

thought experiment: use spark ML to real time prediction

2015-11-10 Thread Andy Davidson
Lets say I have use spark ML to train a linear model. I know I can save and load the model to disk. I am not sure how I can use the model in a real time environment. For example I do not think I can return a ³prediction² to the client using spark streaming easily. Also for some applications the

RE: thought experiment: use spark ML to real time prediction

2015-11-10 Thread Kothuvatiparambil, Viju
ember 10, 2015 1:32 PM To: user @spark Subject: thought experiment: use spark ML to real time prediction Lets say I have use spark ML to train a linear model. I know I can save and load the model to disk. I am not sure how I can use the model in a real time environment. For example I do not think I ca