Thx Yuval I already implemented a different way to create a cluster center and it works great, but I wanted to use the existing Mahout implementation and not develop my own if possible...
I think it is not the Tanimoto distance measure that gives such a centroid - I think it is the way a cluster center is created from points (which currently doesnt depend at all in the distance measure) It creates a center "mathematically legal" for kmeans on Real numbers metrics but doesnt have much sense in my my problem which is a discrete problem i.e. it will not cluster users into clusters by having them read the same documents well enough [Im not interested in the centroid (except for visualization and analysis which is also important) - it is just that the clusters created seems logically wrong to me, and most users were clustered into 1 or 2 clusters with many-many features - too many - with low weights. My new implementation didnt suffer from this problem] I will think on this some more Thx all for your help on this issue On Thu, Jun 21, 2012 at 4:32 PM, Yuval Feinstein <[email protected]>wrote: > Shlomy, I will try to paraphrase what I understood: > a. You are interested in the centroid for its own sake, rather than the > cluster. > b. An ideal centroid will have only zeros or ones as its coordinates, and > thus create a summary of the cluster. > c. The current Tanimoto implementation gives you a different centroid, with > many small coordinate values. > > I do not know enough of the math for K-means, but you could implement your > own distance function, > that will somehow penalize non-integral coordinates/distances. > I am not sure if such a function will generate a converging K-means > algorithm, > but it might be worth a shot. > > If you are looking for summaries of vector sets, you can try matrix > factorizations > (PCA,SVD,NNMF), and see which dominant components come out. > > HTH, > Yuval > > On Thu, Jun 21, 2012 at 2:43 PM, Sean Owen <[email protected]> wrote: > > > Erm, I think I am thinking of canopy clustering. > > > > For k-means, I suppose you could say the choice of the k means isn't > > quite right if k-1 of them are nowhere near most points. I don't know > > how they were chosen. But again maybe not the real issue at heart > > here. > > > > On Thu, Jun 21, 2012 at 12:36 PM, Shlomy Boshy <[email protected]> > > wrote: > > > Interesting - to which thresholds do you refer? > > >
