[
https://issues.apache.org/jira/browse/MAHOUT-843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152124#comment-13152124
]
Paritosh Ranjan commented on MAHOUT-843:
----------------------------------------
Hi Jeff,
I was working on the MapReduce version of the post processor and the solution
you proposed is shaping up. Looks like a really cool solution.
I have a doubt regarding calculation of number of clusters. I think you want to
count the number of centroids produced in the final directory. Am I correct? If
yes, then, is this final directory produced in all the algorithms we want to
support, as you mentioned "Canopy, KMeans, FuzzyK, MeanShift and Dirichlet". If
I have some misconception, then please help to clarify that.
I was also looking into the refactoring that you suggested.
-I will make the code more readable. So, the method naming problem would be
solved.
-Instead of opening and closing SequenceFile.Writer, I can keep them in a map,
for each clusterId, and reuse it. In the end, we can close all of them together
Will this solve the issue?
-Regarding the private fields stuff. I will say that, it looks better to me as
a code point of view. It might be difficult to debug, but I think too many
variables in a method does not account for clean code. I will suggest to keep
it the way it is now. What do you suggest?
> Top Down Clustering
> -------------------
>
> Key: MAHOUT-843
> URL: https://issues.apache.org/jira/browse/MAHOUT-843
> Project: Mahout
> Issue Type: New Feature
> Components: Clustering
> Affects Versions: 0.6
> Reporter: Paritosh Ranjan
> Labels: clustering, patch
> Fix For: 0.6
>
> Attachments: MAHOUT-843-patch, MAHOUT-843-patch-only-postprocessor,
> MAHOUT-843-patch-only-postprocessor-v1,
> MAHOUT-843-patch-only-postprocessor-v2, MAHOUT-843-patch-v1,
> Top-Down-Clustering-patch
>
>
> Top Down Clustering works in multiple steps. The first step is to find
> comparative bigger clusters. The second step is to cluster the bigger chunks
> into meaningful clusters. This can performance while clustering big amount of
> data. And, it also removes the dependency of providing input clusters/numbers
> to the clustering algorithm.
> The "big" is a relative term, as well as the smaller "meaningful" terms. So,
> the control of this "bigger" and "smaller/meaningful" clusters will be
> controlled by the user.
> Which clustering algorithm to be used in the top level and which to use in
> the bottom level can also be selected by the user. Initially, it can be done
> for only one/few clustering algorithms, and later, option can be provided to
> use all the algorithms ( which suits the case ).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
