unsubscribe
unsubscribe smime.p7s Description: S/MIME cryptographic signature
Re: Continuous ML model training in stream mode
Could you please keep your response relevant to the question and the context of the discussion thread? Thanks! On 3/17/22 2:56 PM, Gourav Sengupta wrote: Dear friends, a few years ago, I was in a London meetup seeing Sean (Owen) demonstrate how we can try to predict the gender of individuals who are responding to tweets after accepting privacy agreements, in case I am not wrong. It was real time, it was spectacular, and it was the presentation that set me into data science and its applications. Thanks Sean! :) Regards, Gourav Sengupta On Tue, Mar 15, 2022 at 9:39 PM Artemis User wrote: Thanks Sean! Well, it looks like we have to abandon our structured streaming model to use DStream for this, or do you see possibility to use structured streaming with ml instead of mllib? On 3/15/22 4:51 PM, Sean Owen wrote: There is a streaming k-means example in Spark. https://spark.apache.org/docs/latest/mllib-clustering.html#streaming-k-means On Tue, Mar 15, 2022, 3:46 PM Artemis User wrote: Has anyone done any experiments of training an ML model using stream data? especially for unsupervised models? Any suggestions/references are highly appreciated... - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Re: Continuous ML model training in stream mode
For online recommendation systems, continuous training is needed. :) And we are a living video player, the content is changing every minute, so a real time rec system is the must. On Fri, Mar 18, 2022 at 3:31 AM Sean Owen wrote: > (Thank you, not sure that was me though) > I don't know of plans to expose the streaming impls in ML, as they still > work fine in MLlib and they also don't come up much. Continuous training is > relatively rare, maybe under-appreciated, but rare in practice. > > On Thu, Mar 17, 2022 at 1:57 PM Gourav Sengupta > wrote: > >> Dear friends, >> >> a few years ago, I was in a London meetup seeing Sean (Owen) demonstrate >> how we can try to predict the gender of individuals who are responding to >> tweets after accepting privacy agreements, in case I am not wrong. >> >> It was real time, it was spectacular, and it was the presentation that >> set me into data science and its applications. >> >> Thanks Sean! :) >> >> Regards, >> Gourav Sengupta >> >> >> >> >> On Tue, Mar 15, 2022 at 9:39 PM Artemis User >> wrote: >> >>> Thanks Sean! Well, it looks like we have to abandon our structured >>> streaming model to use DStream for this, or do you see possibility to use >>> structured streaming with ml instead of mllib? >>> >>> On 3/15/22 4:51 PM, Sean Owen wrote: >>> >>> There is a streaming k-means example in Spark. >>> https://spark.apache.org/docs/latest/mllib-clustering.html#streaming-k-means >>> >>> On Tue, Mar 15, 2022, 3:46 PM Artemis User >>> wrote: >>> Has anyone done any experiments of training an ML model using stream data? especially for unsupervised models? Any suggestions/references are highly appreciated... - To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>>
Re: Continuous ML model training in stream mode
we are keeping the training with the input content from a streaming. But the framework is tensorflow not spark. On Wed, Mar 16, 2022 at 4:46 AM Artemis User wrote: > Has anyone done any experiments of training an ML model using stream > data? especially for unsupervised models? Any suggestions/references > are highly appreciated... > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
