I would recommend starting out with something like Spark, but the short answer is that anything that will run inside a yarn container, so the answer is most ML libraries.
Using Spark to train models on the historical store is a good bet, and then using the trained models with model as a service. See https://github.com/apache/metron/tree/master/metron-analytics/metron-maas-service <https://github.com/apache/metron/tree/master/metron-analytics/metron-maas-service> for information on models and some sample boilerplate for deploying your own python based models. You could as some have suggested use spark streaming, but to be honest, the spark ML models are not well suited to streaming use cases, and you would be very much breaking the metron flow rather than benefitting from elements like MaaS (you’d basically be building a 100% custom side project, which would be fine, but you’re missing a lot of the benefits of Metron that way). If you do go down that route I would strong recommend having the output of your streaming jobs feed back into a Metron sensor. To be honest though, you’re much better off training in batch and scoring / inferring via the Model as a Service approach. Simon > On 6 Dec 2017, at 07:45, moshe jarusalem <tuu...@gmail.com> wrote: > > Hi All, > Would you please suggest some documentation about machine learning libraries > can be used in metron architecture? and how ? any examples appretiated. > > regards, >
