In your case, the rank of the Gram matrix is much smaller than the number of samples. All this means that we need to add some kind of regularization to it.
Bertrand On 11/10/2011 04:00 AM, Alejandro Weinstein wrote: > On Mon, Nov 7, 2011 at 12:32 PM, Jacob VanderPlas > <[email protected]> wrote: >> I think, based on this, that KernelPCA is correct as written, except >> that the arpack method should use which='LA' rather than which='LM' >> (thus ignoring any negative eigenvalues). This would fix Alejandro's >> problem. I'll make the change in master. > Does this also affect Hessian LLE and Local Tangent Space Alignment? I > am observing some problems with these two that are solved by using > eigen_solver='dense'. The following code illustrates the problem with > Hessian LLE (I reported the problem with LTSA in a previous post): > > ############################################## > import numpy as np > from sklearn import manifold > > n = 1000; > m = 50; > X = np.random.rand(n,m) > out_dim = 2 > n_neighbors = 10 > Y = manifold.LocallyLinearEmbedding(n_neighbors, out_dim, > eigen_solver='dense', > method='hessian').fit_transform(X) > print "Just computed HLLE using eigen_solver='dense'" > Y = manifold.LocallyLinearEmbedding(n_neighbors, out_dim, > method='hessian').fit_transform(X) > ############################################## > > And this is the output > > Just computed HLLE using eigen_solver='dense' > Traceback (most recent call last): > File "hlle_test.py", line 15, in<module> > method='hessian').fit_transform(X) > File > "/usr/local/lib/python2.6/dist-packages/sklearn/manifold/locally_linear.py", > line 575, in fit_transform > self._fit_transform(X) > File > "/usr/local/lib/python2.6/dist-packages/sklearn/manifold/locally_linear.py", > line 546, in _fit_transform > hessian_tol=self.hessian_tol, modified_tol=self.modified_tol) > File > "/usr/local/lib/python2.6/dist-packages/sklearn/manifold/locally_linear.py", > line 459, in locally_linear_embedding > tol=tol, max_iter=max_iter) > File > "/usr/local/lib/python2.6/dist-packages/sklearn/manifold/locally_linear.py", > line 141, in null_space > tol=tol, maxiter=max_iter) > File "/usr/local/lib/python2.6/dist-packages/sklearn/utils/arpack.py", > line 1488, in eigsh > symmetric=True, tol=tol) > File "/usr/local/lib/python2.6/dist-packages/sklearn/utils/arpack.py", > line 1010, in get_OPinv_matvec > return get_inv_matvec(A, symmetric=symmetric, tol=tol) > File "/usr/local/lib/python2.6/dist-packages/sklearn/utils/arpack.py", > line 1003, in get_inv_matvec > return SpLuInv(M).matvec > File "/usr/local/lib/python2.6/dist-packages/sklearn/utils/arpack.py", > line 896, in __init__ > self.M_lu = splu(M) > File > "/usr/local/lib/python2.6/dist-packages/scipy/sparse/linalg/dsolve/linsolve.py", > line 173, in splu > ilu=False, options=_options) > RuntimeError: Factor is exactly singular > > Alejandro > > ------------------------------------------------------------------------------ > RSA(R) Conference 2012 > Save $700 by Nov 18 > Register now > http://p.sf.net/sfu/rsa-sfdev2dev1 > _______________________________________________ > Scikit-learn-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general ------------------------------------------------------------------------------ RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
