So the problem is to adapt ICA for MR, am i right?
2014/1/7 Dmitriy Lyubimov <[email protected]> > i already looked at fast ICA. while it claims to be parallel, this work > doesn't exactly map it into map reduce (or spark) paradigm and from what i > can recollect still implies outer iterations for fitting principal > component vectors one by one. Which means it probably already is > MR-unfriendly by construction; Spark may show far better promise here but > still a working notes document is required to show how exactly. that's what > i mean. > > > On Tue, Jan 7, 2014 at 1:35 AM, Oleksandr Olgashko < > [email protected] > > wrote: > > > Could you please take a look on this article? > > http://cran.r-project.org/web/packages/fastICA/fastICA.pdf > > I have learned that re-inventing the wheel is wrong for most problems, > and > > usually exists a better solution. However, it often needs some > "grinding", > > so I may research those ways, in case of approval. > > > > About Scala: unfortunately, I have never worked with this language > before, > > but wanted to. I'd like to fill that gap in my skills, but I don't know > > exactly where to start. > > > > > > 2014/1/7 Dmitriy Lyubimov <[email protected]> > > > > > ICA is a very useful technique for dimensionality reduction. I believe > > > Mahout would benefit from it; however challenges are fairly significant > > in > > > terms of proven parallelization technique and acceptable efficacy, > which > > > makes it hard to just "implement" (I am not familiar at this point with > > any > > > concrete work on parallel ICA). So like i said before i am not very > > > hopeful. However, if one never tries, then nothing will get ever done. > > who > > > knows. > > > > > > > > > On Mon, Jan 6, 2014 at 2:18 PM, Isabel Drost-Fromm <[email protected] > > > >wrote: > > > > > > > On Mon, Jan 06, 2014 at 10:40:45PM +0200, Oleksandr Olgashko wrote: > > > > > Returning back to question about theme to work, asked 2 months ago. > > > > > What algorithm should I implement? > > > > > > > > To be quite frank with you: None. Personally I'd rather see > > improvements > > > > (in terms of documentation, integration, stableisation, performance > > > > optimisation) of the existing Mahout source. > > > > > > > > Feel free to take a closer look at the thread concerning "getting > > > > involved" that we had around Christmas last year for inspiration. > > > > > > > > > > > > Isabel > > > > > > > > > > > > > >
