Hi, In this week I have researched on collaborative filtering with implicit feedback data (i.e session data and user behavior). I have followed [1, 2] on this. Even though these papers provide a solution we cannot use them since we cannot use the current implementation in Spark with these.
Also I researched on the possibility of inferring a rating using the session data and create a new data set and push to spark, but almost all the solutions are domain specific. Since the preferences are dependent on item attributes. How should I proceed? Since I have finished implementing the generic recommendation algorithm I can integrate this with ML with a slight design change. Can we have a discussion on this? Thanks, Madawa [1] - https://drive.google.com/file/d/0B5EB-AEf-NyPOHBPRWpRM3hvejA/view?usp=sharing [2] - https://drive.google.com/file/d/0B5EB-AEf-NyPbkVUV2NqZ3R5b0U/view?usp=sharing On 16 June 2015 at 12:00, Madawa Soysa <[email protected]> wrote: > Hi, > > I have developed an sample application to generate recommendations using > Apache Spark MLlib and Java. Project source code can be found in GitHub[1] > <https://github.com/madawas/spark-recommendation>. Output can be taken in > any format and in the current app, I'm printing the output to the console. > The data set used can be found here[2] > <https://drive.google.com/open?id=0B5EB-AEf-NyPflRteEhzNmxFOGpROEFNbWJRa0VTcU5MeEg5WVcya1JIbGM5LVhrU3VXNTQ&authuser=0>. > Please look in to this and provide feedback. > > Next I'm going to modify the application to read data from implicit data > (page views, likes, time spent on page) and infer a rating. I'm referring > to [3] > <https://drive.google.com/open?id=0B5EB-AEf-NyPOHBPRWpRM3hvejA&authuser=0> > on this. > > Also is there any idea on how to extract data from websites? > > > [1] - https://github.com/madawas/spark-recommendation > [2] - > https://drive.google.com/open?id=0B5EB-AEf-NyPflRteEhzNmxFOGpROEFNbWJRa0VTcU5MeEg5WVcya1JIbGM5LVhrU3VXNTQ&authuser=0 > [3] - > https://drive.google.com/open?id=0B5EB-AEf-NyPOHBPRWpRM3hvejA&authuser=0 > > Thanks, > Madawa. > > On 5 June 2015 at 15:27, Madawa Soysa <[email protected]> wrote: > >> Hi, >> >> Had an initial meeting with the mentors about the $subject and discussed >> following matters. >> >> - Data format to be consider in the project >> - Re-Modeling implicit data (mouse clicks, page views) that can be >> extracted from websites to explicit rating scheme. >> - publishing data to DAS and connecting ML with DAS. >> >> Currently I'm developing a sample personalized recommend system to test >> how Apache Spark uses data and how it generates the output model. >> >> Thanks, >> Madawa >> >> On 5 June 2015 at 11:22, Nirmal Fernando <[email protected]> wrote: >> >>> Please also send an update on the meeting to dev@ >>> >>> On Fri, Jun 5, 2015 at 11:19 AM, Nirmal Fernando <[email protected]> >>> wrote: >>> >>>> >>>> https://docs.wso2.com/display/ML100/Integration+with+WSO2+Data+Analytics+Server >>>> >>>> On Thu, Jun 4, 2015 at 3:48 PM, Madawa Soysa <[email protected]> >>>> wrote: >>>> >>>>> OK. Thanks. I will be there. >>>>> >>>>> On 4 June 2015 at 14:55, Nirmal Fernando <[email protected]> wrote: >>>>> >>>>>> great.. confirmed then. >>>>>> >>>>>> On Thu, Jun 4, 2015 at 2:47 PM, Madawa Soysa <[email protected] >>>>>> > wrote: >>>>>> >>>>>>> Yes. Fine with 11 a.m. >>>>>>> >>>>>>> On 4 June 2015 at 14:00, Nirmal Fernando <[email protected]> wrote: >>>>>>> >>>>>>>> 11am ? >>>>>>>> >>>>>>>> On Thu, Jun 4, 2015 at 1:23 PM, Madawa Soysa < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> I'm free in the morning. >>>>>>>>> >>>>>>>>> On 4 June 2015 at 10:52, Nirmal Fernando <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> You are free tomorrow? >>>>>>>>>> >>>>>>>>>> On Thu, Jun 4, 2015 at 10:51 AM, Nirmal Fernando <[email protected] >>>>>>>>>> > wrote: >>>>>>>>>> >>>>>>>>>>> Ok thanks. I'll send you an invite offline. >>>>>>>>>>> >>>>>>>>>>> On Wed, Jun 3, 2015 at 9:44 PM, Madawa Soysa < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Nirmal, >>>>>>>>>>>> >>>>>>>>>>>> I have been playing with the Apache Spark MLlib functions and >>>>>>>>>>>> currently implementing a sample recommendation algorithm using >>>>>>>>>>>> collaborative filtering. I would like to have a meeting as soon as >>>>>>>>>>>> possible >>>>>>>>>>>> before starting to implement the actual component about the >>>>>>>>>>>> workflow and >>>>>>>>>>>> integration details. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Madawa. >>>>>>>>>>>> >>>>>>>>>>>> On 3 June 2015 at 08:30, Nirmal Fernando <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Madawa, >>>>>>>>>>>>> >>>>>>>>>>>>> Please send an update on your work thus far. >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, May 10, 2015 at 2:23 PM, Nirmal Fernando < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Madawa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Welcome to GSoC 15' ! As the first step, please start >>>>>>>>>>>>>> exploring recommendation algorithms of Spark (v1.3) MLLib and we >>>>>>>>>>>>>> would like >>>>>>>>>>>>>> to see a list of supported algorithms and some information on >>>>>>>>>>>>>> each of those. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, May 6, 2015 at 9:31 AM, Madawa Soysa < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Mentors, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> First of all thanks for selecting my proposal on $subject >>>>>>>>>>>>>>> . Even though coding in GSoC starts on 25th May, I would like >>>>>>>>>>>>>>> to have a >>>>>>>>>>>>>>> discussion with mentors and start implementing as soon as >>>>>>>>>>>>>>> possible. It >>>>>>>>>>>>>>> would be great if you can review the proposal and provide >>>>>>>>>>>>>>> feedback. >>>>>>>>>>>>>>> Previous mail thread attached at [1] >>>>>>>>>>>>>>> <http://wso2.markmail.org/search/%5BDev%5D+%5BGSoC-2015%5D+Proposal+11:+Recommendation+Solution+for+WSO2+Machine+Learner+from:%22Madawa+Soysa%22> >>>>>>>>>>>>>>> . >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> http://wso2.markmail.org/search/%5BDev%5D+%5BGSoC-2015%5D+Proposal+11:+Recommendation+Solution+for+WSO2+Machine+Learner+from:%22Madawa+Soysa%22 >>>>>>>>>>>>>>> <http://wso2.markmail.org/search/%5BDev%5D+%5BGSoC-2015%5D+Proposal+11:+Recommendation+Solution+for+WSO2+Machine+Learner+from:%22Madawa+Soysa%22> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Madawa >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> *_________________________________**Madawa Soysa* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Undergraduate, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Department of Computer Science and Engineering, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> University of Moratuwa. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Chairman, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> IEEE Computer Society Student Chapter, >>>>>>>>>>>>>>> University of Moratuwa. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Mobile: +94 75 812 0726 | Email: [email protected] >>>>>>>>>>>>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter >>>>>>>>>>>>>>> <https://twitter.com/madawa_rc> | Tumblr >>>>>>>>>>>>>>> <http://madawas.tumblr.com/> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks & regards, >>>>>>>>>>>>>> Nirmal >>>>>>>>>>>>>> >>>>>>>>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks & regards, >>>>>>>>>>>>> Nirmal >>>>>>>>>>>>> >>>>>>>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> *_________________________________**Madawa Soysa* >>>>>>>>>>>> >>>>>>>>>>>> Undergraduate, >>>>>>>>>>>> >>>>>>>>>>>> Department of Computer Science and Engineering, >>>>>>>>>>>> >>>>>>>>>>>> University of Moratuwa. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Chairman, >>>>>>>>>>>> >>>>>>>>>>>> IEEE Computer Society Student Chapter, >>>>>>>>>>>> University of Moratuwa. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Mobile: +94 75 812 0726 | Email: [email protected] >>>>>>>>>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter >>>>>>>>>>>> <https://twitter.com/madawa_rc> | Tumblr >>>>>>>>>>>> <http://madawas.tumblr.com/> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> Thanks & regards, >>>>>>>>>>> Nirmal >>>>>>>>>>> >>>>>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks & regards, >>>>>>>>>> Nirmal >>>>>>>>>> >>>>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>>>> Mobile: +94715779733 >>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> *_________________________________**Madawa Soysa* >>>>>>>>> >>>>>>>>> Undergraduate, >>>>>>>>> >>>>>>>>> Department of Computer Science and Engineering, >>>>>>>>> >>>>>>>>> University of Moratuwa. >>>>>>>>> >>>>>>>>> >>>>>>>>> Chairman, >>>>>>>>> >>>>>>>>> IEEE Computer Society Student Chapter, >>>>>>>>> University of Moratuwa. >>>>>>>>> >>>>>>>>> >>>>>>>>> Mobile: +94 75 812 0726 | Email: [email protected] >>>>>>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter >>>>>>>>> <https://twitter.com/madawa_rc> | Tumblr >>>>>>>>> <http://madawas.tumblr.com/> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks & regards, >>>>>>>> Nirmal >>>>>>>> >>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>> Mobile: +94715779733 >>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *_________________________________**Madawa Soysa* >>>>>>> >>>>>>> Undergraduate, >>>>>>> >>>>>>> Department of Computer Science and Engineering, >>>>>>> >>>>>>> University of Moratuwa. >>>>>>> >>>>>>> >>>>>>> Chairman, >>>>>>> >>>>>>> IEEE Computer Society Student Chapter, >>>>>>> University of Moratuwa. >>>>>>> >>>>>>> >>>>>>> Mobile: +94 75 812 0726 | Email: [email protected] >>>>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter >>>>>>> <https://twitter.com/madawa_rc> | Tumblr >>>>>>> <http://madawas.tumblr.com/> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & regards, >>>>>> Nirmal >>>>>> >>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>> Mobile: +94715779733 >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *_________________________________**Madawa Soysa* >>>>> >>>>> Undergraduate, >>>>> >>>>> Department of Computer Science and Engineering, >>>>> >>>>> University of Moratuwa. >>>>> >>>>> >>>>> Chairman, >>>>> >>>>> IEEE Computer Society Student Chapter, >>>>> University of Moratuwa. >>>>> >>>>> >>>>> Mobile: +94 75 812 0726 | Email: [email protected] >>>>> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter >>>>> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/> >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>> >>> >>> -- >>> >>> Thanks & regards, >>> Nirmal >>> >>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>> Mobile: +94715779733 >>> Blog: http://nirmalfdo.blogspot.com/ >>> >>> >>> >> >> >> -- >> >> *_________________________________**Madawa Soysa* >> >> Undergraduate, >> >> Department of Computer Science and Engineering, >> >> University of Moratuwa. >> >> >> Chairman, >> >> IEEE Computer Society Student Chapter, >> University of Moratuwa. >> >> >> Mobile: +94 75 812 0726 | Email: [email protected] >> LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter >> <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/> >> > > > > -- > > *_________________________________**Madawa Soysa* > > Undergraduate, > > Department of Computer Science and Engineering, > > University of Moratuwa. > > > Chairman, > > IEEE Computer Society Student Chapter, > University of Moratuwa. > > > Mobile: +94 75 812 0726 | Email: [email protected] > LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter > <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/> > -- *_________________________________**Madawa Soysa* Undergraduate, Department of Computer Science and Engineering, University of Moratuwa. Chairman, IEEE Computer Society Student Chapter, University of Moratuwa. Mobile: +94 75 812 0726 | Email: [email protected] LinkedIn <http://lk.linkedin.com/in/madawasoysa> | Twitter <https://twitter.com/madawa_rc> | Tumblr <http://madawas.tumblr.com/>
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
