Hello Simon,
thanks for the information.
However, why do u affirm that the streaming models are not well suited?
You could as some have suggested use spark streaming, but to be
honest, the spark ML models are not well suited to streaming use cases
Is there a performance problem or how would you justify that phrase?
thanks
Le 07/12/2017 à 13:55, Simon Elliston Ball a écrit :
I would recommend starting out with something like Spark, but the
short answer is that anything that will run inside a yarn container,
so the answer is most ML libraries.
Using Spark to train models on the historical store is a good bet, and
then using the trained models with model as a service.
See
https://github.com/apache/metron/tree/master/metron-analytics/metron-maas-service for
information on models and some sample boilerplate for deploying your
own python based models.
You could as some have suggested use spark streaming, but to be
honest, the spark ML models are not well suited to streaming use
cases, and you would be very much breaking the metron flow rather than
benefitting from elements like MaaS (you’d basically be building a
100% custom side project, which would be fine, but you’re missing a
lot of the benefits of Metron that way). If you do go down that route
I would strong recommend having the output of your streaming jobs feed
back into a Metron sensor. To be honest though, you’re much better off
training in batch and scoring / inferring via the Model as a Service
approach.
Simon
On 6 Dec 2017, at 07:45, moshe jarusalem <[email protected]
<mailto:[email protected]>> wrote:
Hi All,
Would you please suggest some documentation about machine learning
libraries can be used in metron architecture? and how ? any examples
appretiated.
regards,
--
*Martin Andreoni *
PhD. Candidate at GTA/LIP6
UFRJ/UPMC
www.gta.ufrj.br/~martin <http://www.gta.ufrj.br/%7Emartin>
