Yuan Luo <[email protected]> wrote: > Hi, > Does anyone know how I can make GMM parallel the fitting of some moderately > big matrix (say, 390,000 x 400) with 200 components?
I am not sure about GMM code in scikit-learn, but the EM-algorithm for GMMs is very easy to vectorize. There are several ways to do this: 1. Use BLAS and LAPACK to estimate the Gaussians. You can estimate the means with a call to DGEMV and the covariances with a call to DGEMM. To compute Mahalanobis distances use e.g. DPOTRF and DTRTRS (or DTRSM). Then just rely on the LAPACK and BLAS library to do the parallel computing. This works great with Intel MKL, Apple Accelerate Framework or OpenBLAS, but is less effective with ATLAS or ACML. Also it only works on SMP computers. 2. The E-step and M-step each have a loop over the 200 components in the mixture model, and they are called multiple times. Run these loops in parallel and use a barrier synchronization at the end of each loop. You can do this with a threadpool, MPI (e.g. mpi4py) or OpenMP. I would just make sure the functions that estimate or evaluate a Gaussian releases the GIL, and then just use ordinary Python threads. On an SMP computer you should preferably use a single-threaded BLAS library with this solution. 3. Method 1 is inherently data parallel, so you can run it on a GPU with CUDA or OpenCL. If you already have a working solution based on BLAS you can use Nvidia's cuBLAS library. Sturla ------------------------------------------------------------------------------ Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
