Hi,
at first glance, I would say a first step would be to create ML
functions "translated" for the runtime (Spark, Flink, ...).
In the same time, we can create a Beam ML API (generic) that we
translate to the corresponding ML backend (Mahout, Spark ML, ...).
Regards
JB
On 04/20/2016 11:27 AM, Jianfeng Qian wrote:
Hi JB,
I am quite interest about that and I like to contribute to this part.
Does the DSL ML mean we should provide a framwork or API to support mllib of
Spark, FlinkML and Google Cloud Machine Learning.
Or implement algorithms to run at Spark,Flink and DataFlow?
Best Regards,
Jianfeng
________________________________________
From: Jean-Baptiste Onofré <[email protected]>
Sent: Wednesday, April 20, 2016 1:04 AM
To: [email protected]
Subject: Re: Will Beam provide a machine learning API in the future?
Hi Simone,
it sounds good. It's basically what I meant by DSL ML oriented.
We love contribution if you want to work on that with me ;)
Regards
JB
On 04/20/2016 08:27 AM, Simone Robutti wrote:
This would be an interesting feature. We are looking forward to develop ML
integrations on Beam and we are watching what's going on. The idea of a ML
may be interesting as an higher level API or as a proper ML library written
in Beam (pretty much what SAMOA does) but beware to offer a common layer
between different algorithmic implementation because the assumption that
they are consistent in nature and implementation is a big assumption and it
could lead to a lot of design problems for you and usability problems for
the end user.
Il 20/apr/2016 06:16 AM, "Jean-Baptiste Onofré" <[email protected]> ha
scritto:
Hi Jianfreng
As you can see in the "Technical Vision" document:
https://drive.google.com/folderview?id=0B-IhJZh9Ab52OFBVZHpsNjc4eXc&usp=sharing
I proposed "Machine Learning functions support".
It's not the highest priority right now, but it's something that we plan.
Regards
JB
On 04/20/2016 04:23 AM, Jianfeng Qian wrote:
Hi ,
Machine learning become more and more popular today, mllib of Spark,
FlinkML and Google Cloud Machine Learning.
Will Beam provide a machine learning API in the future?
Will anyone have some interest in doing it?
Best Regards,
Jianfeng
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
