Spark’s ML models are primarily batch in their nature. There is talk about
incorporating things like naive bayes and streaming kmeans to structured
streaming (which will require some schema work in metron to make sense). These
are still open issues not seeing a lot of progress in the spark
Simon,
What do you think a good example of python, spark and MaaS would look like?
On December 7, 2017 at 07:56:00, Simon Elliston Ball (
si...@simonellistonball.com) wrote:
I would recommend starting out with something like Spark, but the short
answer is that anything that will run inside a
Hello Simon,
thanks for the information.
However, why do u affirm that the streaming models are not well suited?
You could as some have suggested use spark streaming, but to be
honest, the spark ML models are not well suited to streaming use cases
Is there a performance problem or how would
I would recommend starting out with something like Spark, but the short answer
is that anything that will run inside a yarn container, so the answer is most
ML libraries.
Using Spark to train models on the historical store is a good bet, and then
using the trained models with model as a
Right now, you can look at MaaS, for plugging in machine learning services.
If you want to use spark, and you have it on your cluster, you could write
your own spark drivers and have them pull from the
kakfa topics ( indexing for example ) and run your spark stuff there.
On December 7, 2017 at
