i already looked at fast ICA. while it claims to be parallel, this work doesn't exactly map it into map reduce (or spark) paradigm and from what i can recollect still implies outer iterations for fitting principal component vectors one by one. Which means it probably already is MR-unfriendly by construction; Spark may show far better promise here but still a working notes document is required to show how exactly. that's what i mean.
On Tue, Jan 7, 2014 at 1:35 AM, Oleksandr Olgashko <[email protected] > wrote: > Could you please take a look on this article? > http://cran.r-project.org/web/packages/fastICA/fastICA.pdf > I have learned that re-inventing the wheel is wrong for most problems, and > usually exists a better solution. However, it often needs some "grinding", > so I may research those ways, in case of approval. > > About Scala: unfortunately, I have never worked with this language before, > but wanted to. I'd like to fill that gap in my skills, but I don't know > exactly where to start. > > > 2014/1/7 Dmitriy Lyubimov <[email protected]> > > > ICA is a very useful technique for dimensionality reduction. I believe > > Mahout would benefit from it; however challenges are fairly significant > in > > terms of proven parallelization technique and acceptable efficacy, which > > makes it hard to just "implement" (I am not familiar at this point with > any > > concrete work on parallel ICA). So like i said before i am not very > > hopeful. However, if one never tries, then nothing will get ever done. > who > > knows. > > > > > > On Mon, Jan 6, 2014 at 2:18 PM, Isabel Drost-Fromm <[email protected] > > >wrote: > > > > > On Mon, Jan 06, 2014 at 10:40:45PM +0200, Oleksandr Olgashko wrote: > > > > Returning back to question about theme to work, asked 2 months ago. > > > > What algorithm should I implement? > > > > > > To be quite frank with you: None. Personally I'd rather see > improvements > > > (in terms of documentation, integration, stableisation, performance > > > optimisation) of the existing Mahout source. > > > > > > Feel free to take a closer look at the thread concerning "getting > > > involved" that we had around Christmas last year for inspiration. > > > > > > > > > Isabel > > > > > > > > >
