Ahmed, I can certainly try to answer your questions regarding preprocessing steps. There are basically two mods/additions there compared to original method:
first is to use eigen decomposition instead of SVD on a small hermitian matrix of small dimensions k+p (i.e. just 500x500 symmetric) . Second trick to enable the scale is to produce Y=QR decomposition using blocking and map reduce for matrices exceeding what one could put in RAM. Essentially it is a streaming process with some hierarchical merges. This algorithm builds by induction. Regarding the results, singluar values are actual singular values of the original decomposition so they don't require any postprocessing (aside from taking a sqrt). U and V matrices require simple matrix multiplication, there's not much new here, Mahout's DistributedRowMatrix does it too (except in case of SSVD, this can be done as map-only process due to extremely small size of singular vector matrix of the reduced matrix). The forumulas used are all there, i can certainly try to interpret them for you, i just need to understand where you got stuck. -d On Thu, Jan 27, 2011 at 6:30 AM, ahmed.nagy <[email protected]>wrote: > >
