Re: Running MAAS in batch
You model is really just a function that you wrap in a REST service in order to deploy in MaaS. In the case of something like spark, you would just wrap it in a udf instead of wrapping it in a REST service, at that point, applying it in batch is just a case of a simple dataframe query. On Fri, 16 Nov 2018 at 15:51, deepak kumar wrote: > Simon, > Can you elaborate more on this: > ' > > *wrapped up in a batch engine like Spark to takeadvantage of more > efficient "mass" scoring.* > ' > How the mass model wrapped in spark can take advantage of mass scoring? > > Thanks > Deepak > > On Fri, Nov 16, 2018 at 9:15 PM Otto Fowler > wrote: > >> That may be the best MAAS explanation I’ve seen Simon. >> >> >> On November 16, 2018 at 10:28:57, Simon Elliston Ball ( >> si...@simonellistonball.com) wrote: >> >> MaaS is designed to wrap model inference (scoring) an event at a time, >> via a REST api. As such, running it batch doesn't make a lot of sense, >> since each message would be processed individually. Most of the models >> you're likely to run in MaaS however, are also likely to be easily >> batchable, and are probable better wrapped up in a batch engine like Spark >> to take advantage of more efficient "mass" scoring. >> >> Simon >> >> On Fri, 16 Nov 2018 at 15:18, deepak kumar wrote: >> >>> Hi All >>> Right now MAAS supports running the model against real time events being >>> streamed into metron platform. >>> Is there any way to run the models deployed in MAAS on the batch events >>> / data that have been indexed into hdfs ? >>> If anyone have tried this batch model , please share some insights. >>> Thanks >>> Deepak. >>> >>> >> >> -- >> -- >> simon elliston ball >> @sireb >> >> -- -- simon elliston ball @sireb
Re: Running MAAS in batch
Simon, Can you elaborate more on this: ' *wrapped up in a batch engine like Spark to takeadvantage of more efficient "mass" scoring.* ' How the mass model wrapped in spark can take advantage of mass scoring? Thanks Deepak On Fri, Nov 16, 2018 at 9:15 PM Otto Fowler wrote: > That may be the best MAAS explanation I’ve seen Simon. > > > On November 16, 2018 at 10:28:57, Simon Elliston Ball ( > si...@simonellistonball.com) wrote: > > MaaS is designed to wrap model inference (scoring) an event at a time, via > a REST api. As such, running it batch doesn't make a lot of sense, since > each message would be processed individually. Most of the models you're > likely to run in MaaS however, are also likely to be easily batchable, and > are probable better wrapped up in a batch engine like Spark to take > advantage of more efficient "mass" scoring. > > Simon > > On Fri, 16 Nov 2018 at 15:18, deepak kumar wrote: > >> Hi All >> Right now MAAS supports running the model against real time events being >> streamed into metron platform. >> Is there any way to run the models deployed in MAAS on the batch events / >> data that have been indexed into hdfs ? >> If anyone have tried this batch model , please share some insights. >> Thanks >> Deepak. >> >> > > -- > -- > simon elliston ball > @sireb > >
Re: Running MAAS in batch
MaaS is designed to wrap model inference (scoring) an event at a time, via a REST api. As such, running it batch doesn't make a lot of sense, since each message would be processed individually. Most of the models you're likely to run in MaaS however, are also likely to be easily batchable, and are probable better wrapped up in a batch engine like Spark to take advantage of more efficient "mass" scoring. Simon On Fri, 16 Nov 2018 at 15:18, deepak kumar wrote: > Hi All > Right now MAAS supports running the model against real time events being > streamed into metron platform. > Is there any way to run the models deployed in MAAS on the batch events / > data that have been indexed into hdfs ? > If anyone have tried this batch model , please share some insights. > Thanks > Deepak. > > -- -- simon elliston ball @sireb
