[
https://issues.apache.org/jira/browse/MAHOUT-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660103#comment-13660103
]
Suneel Marthi commented on MAHOUT-1218:
---------------------------------------
What triggered this was a bunch of tests that I was running with Dan this
morning on about 4 * 10^6 data points. We started with k = 60, km = (k log
n)/(# of mappers = 144) (~ 913). We were consistently seeing the issue that Dan
had reported in Mahout-1217.
Dan had suggested to set km to be the same value as k (which through this
error). But now I realize that it should have been km = n log k and this error
is expected behavior.
I will mark this as invalid.
> Streamimg k-means fails when the number of clusters specified is <= estimated
> map clusters
> ------------------------------------------------------------------------------------------
>
> Key: MAHOUT-1218
> URL: https://issues.apache.org/jira/browse/MAHOUT-1218
> Project: Mahout
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 0.8
> Reporter: Suneel Marthi
> Assignee: Suneel Marthi
>
> Running Streaming k-means with CosineDistanceMeasure, Fast Projection Search,
> number of clusters k= 60, number of estimated map clsuters -km = 60.
> {Code}
> Exception in thread "main" java.lang.IllegalArgumentException: Invalid number
> of estimated map clusters; There must be more than the final number of
> clusters (k log n vs k)
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
> at
> org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansDriver.configureOptionsForWorkers(StreamingKMeansDriver.java:327)
> at
> org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansDriver.configureOptionsForWorkers(StreamingKMeansDriver.java:280)
> at
> org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansDriver.run(StreamingKMeansDriver.java:227)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at
> org.apache.mahout.clustering.streaming.mapreduce.StreamingKMeansDriver.main(StreamingKMeansDriver.java:472)
> {Code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
