Re: Adding ML to Ignite, IGNITE-4572
Oleg, Perfect, thanks! Both Prachi and I reviewed your documentation and put it on the readme. IGNITE-4964 is closed. Please, put some extra efforts as a part of Ignite 2.0 release considering latests review notes in this ticket: https://issues.apache.org/jira/browse/IGNITE-5018 <https://issues.apache.org/jira/browse/IGNITE-5018> — Denis > On Apr 18, 2017, at 10:20 AM, oignatenko <oignate...@gridgain.com> wrote: > > Denis, > > As far as I can tell renames are now finalized in the way you proposed > yesterday. I re-run my checks for build, module unit tests and examples > (everything went fine) and updated draft doc for readme.io accordingly. > > You can find most recent version of the draft doc attached to JIRA > IGNITE-4964 > > regards, Oleg > > > Denis Magda wrote >> Thanks Oleg, I’ll review the doc a bit later. >> >> Yuri, I’ve reopened IGNITE-5000. Please consider my latest notes and >> prepare one more pull-request: >> https://issues.apache.org/jira/browse/IGNITE-5000 >> https://issues.apache.org/jira/browse/IGNITE-5000; >> >> — >> Denis >> >>> On Apr 17, 2017, at 8:48 AM, oignatenko > >> oignatenko@ > >> wrote: >>> >>> Denis, I prepared preliminary draft doc for readme.io >>> http://readme.io/;, it reflects proposed >>> rename. You can find it attached to JIRA IGNITE-4964 >>> >>> With regards to merge, per my reading of recent mail from Yury it is done >>> except for module rename which is on its way to master now. >>> >>> regards, Oleg >>> >>> >>> Denis Magda-2 wrote >>>> Oleg, Nikita, Yuri, >>>> >>>> Frankly, I don’t like how the module is named presently which is >>>> 'ignite-math’. >>>> >>>> I propose to rename it to ‘ignite-ml’. Presently, it holds a small >>>> fraction of all ML functionality we might have in the future and this is >>>> why, thinking of future, it will be a right decision to name it >>>> accordingly. >>>> >>>> Any concerns? >>>> >>>> BTW, have the module been fully merged? I want to try to build it and >>>> check the examples. >>>> >>>> — >>>> Denis >>>> >>>>> On Apr 14, 2017, at 7:05 AM, oignatenko >>> >>>> oignatenko@ >>> >>>> wrote: >>>>> >>>>> Thank you Denis, that sounds like a good plan. I am working on it now. >>>>> I >>>>> didn't reassign IGNITE-4964, just set a watch on it. Will comment on it >>>>> when >>>>> I have concrete details to share. >>>>> >>>>> Side note my build check completed successfully, although it took quite >>>>> a >>>>> bit of time: I cleaned my local maven repo to make sure that things >>>>> really >>>>> work from the clean state. >>>>> >>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16677.html >>>>> Sent from the Apache Ignite Developers mailing list archive at >>>>> Nabble.com. >>> >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16758.html >>> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16758.html; >>> Sent from the Apache Ignite Developers mailing list archive at Nabble.com >>> http://nabble.com/;. > > > > > > -- > View this message in context: > http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16878.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Denis, As far as I can tell renames are now finalized in the way you proposed yesterday. I re-run my checks for build, module unit tests and examples (everything went fine) and updated draft doc for readme.io accordingly. You can find most recent version of the draft doc attached to JIRA IGNITE-4964 regards, Oleg Denis Magda wrote > Thanks Oleg, I’ll review the doc a bit later. > > Yuri, I’ve reopened IGNITE-5000. Please consider my latest notes and > prepare one more pull-request: > https://issues.apache.org/jira/browse/IGNITE-5000 > https://issues.apache.org/jira/browse/IGNITE-5000; > > — > Denis > >> On Apr 17, 2017, at 8:48 AM, oignatenko > oignatenko@ > wrote: >> >> Denis, I prepared preliminary draft doc for readme.io >> http://readme.io/;, it reflects proposed >> rename. You can find it attached to JIRA IGNITE-4964 >> >> With regards to merge, per my reading of recent mail from Yury it is done >> except for module rename which is on its way to master now. >> >> regards, Oleg >> >> >> Denis Magda-2 wrote >>> Oleg, Nikita, Yuri, >>> >>> Frankly, I don’t like how the module is named presently which is >>> 'ignite-math’. >>> >>> I propose to rename it to ‘ignite-ml’. Presently, it holds a small >>> fraction of all ML functionality we might have in the future and this is >>> why, thinking of future, it will be a right decision to name it >>> accordingly. >>> >>> Any concerns? >>> >>> BTW, have the module been fully merged? I want to try to build it and >>> check the examples. >>> >>> — >>> Denis >>> >>>> On Apr 14, 2017, at 7:05 AM, oignatenko >> >>> oignatenko@ >> >>> wrote: >>>> >>>> Thank you Denis, that sounds like a good plan. I am working on it now. >>>> I >>>> didn't reassign IGNITE-4964, just set a watch on it. Will comment on it >>>> when >>>> I have concrete details to share. >>>> >>>> Side note my build check completed successfully, although it took quite >>>> a >>>> bit of time: I cleaned my local maven repo to make sure that things >>>> really >>>> work from the clean state. >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16677.html >>>> Sent from the Apache Ignite Developers mailing list archive at >>>> Nabble.com. >> >> >> >> >> >> -- >> View this message in context: >> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16758.html >> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16758.html; >> Sent from the Apache Ignite Developers mailing list archive at Nabble.com >> http://nabble.com/;. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16878.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Thanks Oleg, I’ll review the doc a bit later. Yuri, I’ve reopened IGNITE-5000. Please consider my latest notes and prepare one more pull-request: https://issues.apache.org/jira/browse/IGNITE-5000 <https://issues.apache.org/jira/browse/IGNITE-5000> — Denis > On Apr 17, 2017, at 8:48 AM, oignatenko <oignate...@gridgain.com> wrote: > > Denis, I prepared preliminary draft doc for readme.io <http://readme.io/>, it > reflects proposed > rename. You can find it attached to JIRA IGNITE-4964 > > With regards to merge, per my reading of recent mail from Yury it is done > except for module rename which is on its way to master now. > > regards, Oleg > > > Denis Magda-2 wrote >> Oleg, Nikita, Yuri, >> >> Frankly, I don’t like how the module is named presently which is >> 'ignite-math’. >> >> I propose to rename it to ‘ignite-ml’. Presently, it holds a small >> fraction of all ML functionality we might have in the future and this is >> why, thinking of future, it will be a right decision to name it >> accordingly. >> >> Any concerns? >> >> BTW, have the module been fully merged? I want to try to build it and >> check the examples. >> >> — >> Denis >> >>> On Apr 14, 2017, at 7:05 AM, oignatenko > >> oignatenko@ > >> wrote: >>> >>> Thank you Denis, that sounds like a good plan. I am working on it now. I >>> didn't reassign IGNITE-4964, just set a watch on it. Will comment on it >>> when >>> I have concrete details to share. >>> >>> Side note my build check completed successfully, although it took quite a >>> bit of time: I cleaned my local maven repo to make sure that things >>> really >>> work from the clean state. >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16677.html >>> Sent from the Apache Ignite Developers mailing list archive at >>> Nabble.com. > > > > > > -- > View this message in context: > http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16758.html > > <http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16758.html> > Sent from the Apache Ignite Developers mailing list archive at Nabble.com > <http://nabble.com/>.
Re: Adding ML to Ignite, IGNITE-4572
Denis, I prepared preliminary draft doc for readme.io, it reflects proposed rename. You can find it attached to JIRA IGNITE-4964 With regards to merge, per my reading of recent mail from Yury it is done except for module rename which is on its way to master now. regards, Oleg Denis Magda-2 wrote > Oleg, Nikita, Yuri, > > Frankly, I don’t like how the module is named presently which is > 'ignite-math’. > > I propose to rename it to ‘ignite-ml’. Presently, it holds a small > fraction of all ML functionality we might have in the future and this is > why, thinking of future, it will be a right decision to name it > accordingly. > > Any concerns? > > BTW, have the module been fully merged? I want to try to build it and > check the examples. > > — > Denis > >> On Apr 14, 2017, at 7:05 AM, oignatenko > oignatenko@ > wrote: >> >> Thank you Denis, that sounds like a good plan. I am working on it now. I >> didn't reassign IGNITE-4964, just set a watch on it. Will comment on it >> when >> I have concrete details to share. >> >> Side note my build check completed successfully, although it took quite a >> bit of time: I cleaned my local maven repo to make sure that things >> really >> work from the clean state. >> >> >> >> -- >> View this message in context: >> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16677.html >> Sent from the Apache Ignite Developers mailing list archive at >> Nabble.com. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16758.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Denis, Yes, this module already in apache master branch. Also I created ticket for module renaming, so we could discuss this renaming in IGNITE-5000 -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16743.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Oleg, Nikita, Yuri, Frankly, I don’t like how the module is named presently which is 'ignite-math’. I propose to rename it to ‘ignite-ml’. Presently, it holds a small fraction of all ML functionality we might have in the future and this is why, thinking of future, it will be a right decision to name it accordingly. Any concerns? BTW, have the module been fully merged? I want to try to build it and check the examples. — Denis > On Apr 14, 2017, at 7:05 AM, oignatenko <oignate...@gridgain.com> wrote: > > Thank you Denis, that sounds like a good plan. I am working on it now. I > didn't reassign IGNITE-4964, just set a watch on it. Will comment on it when > I have concrete details to share. > > Side note my build check completed successfully, although it took quite a > bit of time: I cleaned my local maven repo to make sure that things really > work from the clean state. > > > > -- > View this message in context: > http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16677.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Thank you Denis, that sounds like a good plan. I am working on it now. I didn't reassign IGNITE-4964, just set a watch on it. Will comment on it when I have concrete details to share. Side note my build check completed successfully, although it took quite a bit of time: I cleaned my local maven repo to make sure that things really work from the clean state. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16677.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Hi Oleg, That’s great, thanks. I’ve created a JIRA ticket for this task and assigned on Nikita for now. Please create an account for yourself and reassign the ticket if needed: https://issues.apache.org/jira/browse/IGNITE-4964 Plus, there is an invisible page on readme.io <http://readme.io/> created for you where you can document everything right away: https://dash.readme.io/project/apacheignite/v1.9/docs/distributed-algebra I’ll grant you editor rights to the readme today, you’ll receive a notification. — Denis > On Apr 13, 2017, at 9:33 AM, oignatenko <oignate...@gridgain.com> wrote: > > Yury, Denis - me and Nikita are going to take care of readme.io docs. > > I plan to start on this tomorrow, will keep you posted on the progress. > > Current plan is to briefly introduce new package and examples and explain > how to run examples and possibly unit tests. > > With regards to build details, this will depend on how accurate is the > description in project readme files: I am currently running build check to > verify everything in details. If readme files are okay we will likely just > refer readers to these for more details. In case if there are any > discrepancies we will provide concrete build details at readme.io. > > If you have anything to add or correct in above plan please let me know. > > regards, Oleg > > Yury Babak wrote >> As far as I know Nikita wants to provide this documentation. > > > > > > -- > View this message in context: > http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16603.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Yury, Denis - me and Nikita are going to take care of readme.io docs. I plan to start on this tomorrow, will keep you posted on the progress. Current plan is to briefly introduce new package and examples and explain how to run examples and possibly unit tests. With regards to build details, this will depend on how accurate is the description in project readme files: I am currently running build check to verify everything in details. If readme files are okay we will likely just refer readers to these for more details. In case if there are any discrepancies we will provide concrete build details at readme.io. If you have anything to add or correct in above plan please let me know. regards, Oleg Yury Babak wrote > As far as I know Nikita wants to provide this documentation. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16603.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
As far as I know Nikita wants to provide this documentation. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16542.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Guys, a bit more details about the current state of ML lib and how we plan to release it. Currently IgniteML is separate module and use java8. We depends only on ignite-core module. So we think that we could release lib as sources. It should be much easier than provide our lib binary. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16532.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Sorry for the late reply. Sure, you're always welcomed to join our community and participate in the development of modules of interest. As for ML, please keep an eye on this discussion where will break down ML activities a bit later. Denis On Wednesday, April 5, 2017, anantbietec <anantbie...@gmail.com> wrote: > I am very enthusiatic about this project but sir i have not submitted > PROPOSAL FOR THIS IN gsoc 2017 can i still be allowed to do this project > > As the reply comes very late i didnt know what are the tasks to do in this > project . So i didnt prepare the proposal . > > Can we still have the discussion and assign tasks to me so that i can > contribute to the project and also tell me upto now what has been done so > that i am also on the same page. > > Thanks > Anant > > > > -- > View this message in context: http://apache-ignite- > developers.2346864.n4.nabble.com/Adding-ML-to-Ignite- > IGNITE-4572-tp13936p16190.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com. >
Re: Adding ML to Ignite, IGNITE-4572
I am very enthusiatic about this project but sir i have not submitted PROPOSAL FOR THIS IN gsoc 2017 can i still be allowed to do this project As the reply comes very late i didnt know what are the tasks to do in this project . So i didnt prepare the proposal . Can we still have the discussion and assign tasks to me so that i can contribute to the project and also tell me upto now what has been done so that i am also on the same page. Thanks Anant -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16190.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
These are some suggestions from Nikita that were sent to Alper privately that might be useful for other candidates: Alper, Here are some of the immediate tasks we'll be working on past 2.0 release (and you are 100% welcome to tackle them if you find them interesting): (1) Linear regression algorithm on super-large distributed sparse dataset (2) Logistic regression algorithm on super-large distributed sparse dataset Thanks! > On Apr 3, 2017, at 12:21 AM, anantbietec <anantbie...@gmail.com> wrote: > > yeah thats good but GSOC application last till today only so any suggestion > how to write the proposal or what need to be included in the proposal. > > > > -- > View this message in context: > http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16070.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
yeah thats good but GSOC application last till today only so any suggestion how to write the proposal or what need to be included in the proposal. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16070.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Thank you for your reply. I will be watching this thread. However, GSOC applications are due Monday. So, some suggestions for what to write on the proposal would be great, if pull request will be later than Monday. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p15902.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Alper - we are very close to "pull request" our preliminary work into ignite 2.0 (finger crossed it will make it into it). Once we have it available in the main ignite 2.0 branch there will be plenty of interesting tasks to tackle. Stay tuned on this thread! -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p15901.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Hi, I am Alper. I am a computer engineering and mathematics (as double major) student with great interest in machine learning. I wish to participate in GSOC - 2017 and I've been interested in this "issue" for a while. I think that my skills fits this project and that I can really be useful and contribute. Can you please help me about where I need to start from? Thank you. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p15871.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
+100 Great feature! I too would like to see a list of tickets required for the release. D. On Tue, Mar 28, 2017 at 11:24 AM, Denis Magda <dma...@apache.org> wrote: > Unbelievable, that’s astonishing we can already release the distributed > algebra support in the beta state. Please share the tasks you’re planning > to work on right after the release. > > — > Denis > > > On Mar 28, 2017, at 11:12 AM, nivanov <nivano...@gmail.com> wrote: > > > > Just a quick follow up... > > > > We've made some good progress on the initial code implementation in the > last > > couple of months and I think we'll make it on time to release a early > "beta" > > version of the IgniteML (a.k.a distributed math) for 2.0 release. > > > > > > > > -- > > View this message in context: http://apache-ignite- > developers.2346864.n4.nabble.com/Adding-ML-to-Ignite- > IGNITE-4572-tp13936p15858.html > > Sent from the Apache Ignite Developers mailing list archive at > Nabble.com. > >
Re: Adding ML to Ignite, IGNITE-4572
Unbelievable, that’s astonishing we can already release the distributed algebra support in the beta state. Please share the tasks you’re planning to work on right after the release. — Denis > On Mar 28, 2017, at 11:12 AM, nivanov <nivano...@gmail.com> wrote: > > Just a quick follow up... > > We've made some good progress on the initial code implementation in the last > couple of months and I think we'll make it on time to release a early "beta" > version of the IgniteML (a.k.a distributed math) for 2.0 release. > > > > -- > View this message in context: > http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p15858.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Adding ML to Ignite, IGNITE-4572
Just a quick follow up... We've made some good progress on the initial code implementation in the last couple of months and I think we'll make it on time to release a early "beta" version of the IgniteML (a.k.a distributed math) for 2.0 release. -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p15858.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Adding ML to Ignite, IGNITE-4572
Igniters, I'd like to kick off the conversation on adding ML capabilities to Ignite as it relates to IGNITE-4572. I started off some preliminary steps on this ticket and would like to invite anyone interesting in implementing distributed core algebra on top of Ignite to join me in this work. There's plenty to contribute from ideas and math, to implementation, tests, and documentation. Background on IGNITE-4572. Throughout many conferences last year I have heard the sentiment that one of the functional areas that is badly missing in Ignite is a library of distributed Machine Learning algorithms. In fact, it was frequently noted as the only reason not to finally migrate from Spark to Ignite... ML, in general, is a pretty large undertaking and there are more than a few ways to approach this in the context of Ignite. One idea I'd like to chew on and prototype is to look close at Apache Mahout ( http://mahout.apache.org/). After a deep look at Mahout codebase I don't believe the outright code-level integration is feasible at this points - but we can certainly borrow as many ideas from it as possible. Thanks! -- Nikita Ivanov
