Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-29 Thread Nelle Varoquaux
On 28 March 2013 23:24, Lars Buitinck wrote: > On Thu, Mar 28, 2013 at 2:35 PM, Nelle Varoquaux > wrote: > > But in general, I don't think we can "force" the user to use sparse > > matrices. They are an absolute pain to work with because of the > > inconsistencies of interface with ndarray and c

Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Mathieu Blondel
On Fri, Mar 29, 2013 at 7:24 AM, Lars Buitinck wrote: > I know the "need to rewrite scipy.sparse" itch. I for one would be > very excited if you were to volunteer ;) I almost started this project twice but then I realized I would shoot me in the foot if I started it alone and gave up. If we do i

Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Lars Buitinck
On Thu, Mar 28, 2013 at 2:35 PM, Nelle Varoquaux wrote: > But in general, I don't think we can "force" the user to use sparse > matrices. They are an absolute pain to work with because of the > inconsistencies of interface with ndarray and conversion between sparse and > dense can be time consumin

Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Kenneth C. Arnold
On Thu, Mar 28, 2013 at 2:35 PM, Nelle Varoquaux wrote: > But in general, I don't think we can "force" the user to use sparse > matrices. They are an absolute pain to work with because of the > inconsistencies of interface with ndarray and conversion between sparse and > dense can be time consumin

Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Nelle Varoquaux
On 28 March 2013 18:19, Jacob Vanderplas wrote: > On Thu, Mar 28, 2013 at 10:10 AM, Lars Buitinck wrote: > >> 2013/3/28 Mathieu Blondel : >> > Encoding missing values with np.nan doesn't scale to very >> > high-dimensional problems with mostly missing values. >> > Personally, for encoding missing

Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Jacob Vanderplas
On Thu, Mar 28, 2013 at 10:10 AM, Lars Buitinck wrote: > 2013/3/28 Mathieu Blondel : > > Encoding missing values with np.nan doesn't scale to very > > high-dimensional problems with mostly missing values. > > Personally, for encoding missing data, I just use sparse matrices. > > Values which are

Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Lars Buitinck
2013/3/28 Mathieu Blondel : > Encoding missing values with np.nan doesn't scale to very > high-dimensional problems with mostly missing values. > Personally, for encoding missing data, I just use sparse matrices. > Values which are actually zero can be stored explicitly in the .data > attribute. +

Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Mathieu Blondel
On Fri, Mar 29, 2013 at 12:57 AM, Nelle Varoquaux wrote: > We need to find a uniform way over the whole scikit to indicate missing > data. Hence, 0 cannot be how missing data is spotted. > A solution would be to use "Nan" but it is not very satisfying either, as > this could lead to think there is

Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Gael Varoquaux
On Thu, Mar 28, 2013 at 04:57:37PM +0100, Nelle Varoquaux wrote: > Maybe we should add an argument named missing, with how the missing data is > indicated in the matrices ? For example, the signature of the MDS, using nan > as > missing data would be something like: >     mds.fit(X, missing=np.na

Re: [Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Nelle Varoquaux
Hi Terry, We need to find a uniform way over the whole scikit to indicate missing data. Hence, 0 cannot be how missing data is spotted. A solution would be to use "Nan" but it is not very satisfying either, as this could lead to think there is missing data, while there isn't. Maybe we should add

[Scikit-learn-general] algorithm solve classical MDS with SVD

2013-03-28 Thread Terry Peng
Hi all, I was discussing with Nelle to add an algorithm to solve the classical MDS with svd. but one thing we don't sure is how to check missing data so we can fall back to SMACOF in that case. my idea is to check if there are any 0 in non-diagonal elements. what do you think? Thanks & Regar