[
https://issues.apache.org/jira/browse/MAHOUT-504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208190#comment-13208190
]
qiang xu edited comment on MAHOUT-504 at 2/15/12 2:20 AM:
----------------------------------------------------------
I think there is nothing wrong with the path.
Because the /user/root/examples/bin/work/clusters is generated by kmeans
example.
All my steps are:
./bin/mahout org.apache.lucene.benchmark.utils.ExtractReuters
./examples/bin/work/reuters-sgm/ ./examples/bin/work/reuters-out/
./bin/mahout seqdirectory -i ./examples/bin/work/reuters-out/ -o
./examples/bin/work/reuters-out-seqdir -c UTF-8 -chunk 5 -ow
./bin/mahout seq2sparse -i ./examples/bin/work/reuters-out-seqdir/ -o
./examples/bin/work/reuters-out-seqdir-sparse
./bin/mahout kmeans -i
./examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors/ -c
./examples/bin/work/clusters -o ./examples/bin/work/reuters-kmeans -x 10 -k 20
-ow
./bin/mahout clusterdump -s examples/bin/work/reuters-kmeans/clusters-10 -d
examples/bin/work/reuters-out-seqdir-sparse/dictionary.file-0 -dt sequencefile
-b 100 -n 20
I have also tested with aboosolute path of hdfs as following:
[root@qxutest mahout-distribution-0.5]# hadoop fs -ls
/user/root/examples/bin/work/
Found 4 items
drwxr-xr-x - root supergroup 0 2012-02-14 20:55
/user/root/examples/bin/work/clusters
drwxr-xr-x - root supergroup 0 2012-02-14 20:56
/user/root/examples/bin/work/reuters-kmeans
drwxr-xr-x - root supergroup 0 2012-02-14 20:29
/user/root/examples/bin/work/reuters-out-seqdir
drwxr-xr-x - root supergroup 0 2012-02-14 20:32
/user/root/examples/bin/work/reuters-out-seqdir-sparse
[root@qxutest mahout-distribution-0.5]# hadoop fs -ls
/user/root/examples/bin/work/clusters
Found 1 items
-rw-r--r-- 2 root supergroup 139 2012-02-14 20:55
/user/root/examples/bin/work/clusters/part-randomSeed
[root@qxutest mahout-distribution-0.5]# ./bin/mahout kmeans -i
/user/root/examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors/ -c
/user/root/examples/bin/work/clusters -o
/user/root/examples/bin/work/reuters-kmeans -x 10 -ow
Running on hadoop, using HADOOP_HOME=/data/hadoop_cluster/hadoop-0.20.2/
HADOOP_CONF_DIR=/data/hadoop_cluster/hadoop-0.20.2/conf/
12/02/15 10:32:25 INFO common.AbstractJob: Command line arguments:
{--clusters=/user/root/examples/bin/work/clusters, --convergenceDelta=0.5,
--distanceMeasure=org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure,
--endPhase=2147483647,
--input=/user/root/examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors/,
--maxIter=10, --method=mapreduce,
--output=/user/root/examples/bin/work/reuters-kmeans, --overwrite=null,
--startPhase=0, --tempDir=temp}
12/02/15 10:32:25 INFO common.HadoopUtil: Deleting
/user/root/examples/bin/work/reuters-kmeans
12/02/15 10:32:25 INFO kmeans.KMeansDriver: Input:
/user/root/examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors Clusters
In: /user/root/examples/bin/work/clusters Out:
/user/root/examples/bin/work/reuters-kmeans Distance:
org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure
12/02/15 10:32:25 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10
num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/02/15 10:32:25 INFO kmeans.KMeansDriver: K-Means Iteration 1
12/02/15 10:32:26 INFO input.FileInputFormat: Total input paths to process : 1
12/02/15 10:32:27 INFO mapred.JobClient: Running job: job_201202131515_0123
12/02/15 10:32:28 INFO mapred.JobClient: map 0% reduce 0%
12/02/15 10:32:38 INFO mapred.JobClient: Task Id :
attempt_201202131515_0123_m_000000_0, Status : FAILED
java.lang.IllegalStateException: No clusters found. Check your -c path.
at
org.apache.mahout.clustering.kmeans.KMeansMapper.setup(KMeansMapper.java:60)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Also without ./
./bin/mahout kmeans -i
examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors/ -c
examples/bin/work/clusters -o examples/bin/work/reuters-kmeans -x 10 -ow
Running on hadoop, using HADOOP_HOME=/data/hadoop_cluster/hadoop-0.20.2/
HADOOP_CONF_DIR=/data/hadoop_cluster/hadoop-0.20.2/conf/
12/02/15 10:38:36 INFO common.AbstractJob: Command line arguments:
{--clusters=examples/bin/work/clusters, --convergenceDelta=0.5,
--distanceMeasure=org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure,
--endPhase=2147483647,
--input=examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors/,
--maxIter=10, --method=mapreduce, --output=examples/bin/work/reuters-kmeans,
--overwrite=null, --startPhase=0, --tempDir=temp}
12/02/15 10:38:37 INFO common.HadoopUtil: Deleting
examples/bin/work/reuters-kmeans
12/02/15 10:38:37 INFO kmeans.KMeansDriver: Input:
examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors Clusters In:
examples/bin/work/clusters Out: examples/bin/work/reuters-kmeans Distance:
org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure
12/02/15 10:38:37 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10
num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/02/15 10:38:37 INFO kmeans.KMeansDriver: K-Means Iteration 1
12/02/15 10:38:37 INFO input.FileInputFormat: Total input paths to process : 1
12/02/15 10:38:38 INFO mapred.JobClient: Running job: job_201202131515_0124
12/02/15 10:38:39 INFO mapred.JobClient: map 0% reduce 0%
12/02/15 10:38:50 INFO mapred.JobClient: Task Id :
attempt_201202131515_0124_m_000000_0, Status : FAILED
java.lang.IllegalStateException: No clusters found. Check your -c path.
at
org.apache.mahout.clustering.kmeans.KMeansMapper.setup(KMeansMapper.java:60)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
was (Author: skaterxu):
I think there is nothing wrong with the path.
Because the /user/root/examples/bin/work/clusters is generated by kmeans
example.
All my steps are:
./bin/mahout org.apache.lucene.benchmark.utils.ExtractReuters
./examples/bin/work/reuters-sgm/ ./examples/bin/work/reuters-out/
./bin/mahout seqdirectory -i ./examples/bin/work/reuters-out/ -o
./examples/bin/work/reuters-out-seqdir -c UTF-8 -chunk 5 -ow
./bin/mahout seq2sparse -i ./examples/bin/work/reuters-out-seqdir/ -o
./examples/bin/work/reuters-out-seqdir-sparse
./bin/mahout kmeans -i
./examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors/ -c
./examples/bin/work/clusters -o ./examples/bin/work/reuters-kmeans -x 10 -k 20
-ow
./bin/mahout clusterdump -s examples/bin/work/reuters-kmeans/clusters-10 -d
examples/bin/work/reuters-out-seqdir-sparse/dictionary.file-0 -dt sequencefile
-b 100 -n 20
I have also tested with aboosolute path of hdfs as following:
[root@qxutest mahout-distribution-0.5]# hadoop fs -ls
/user/root/examples/bin/work/
Found 4 items
drwxr-xr-x - root supergroup 0 2012-02-14 20:55
/user/root/examples/bin/work/clusters
drwxr-xr-x - root supergroup 0 2012-02-14 20:56
/user/root/examples/bin/work/reuters-kmeans
drwxr-xr-x - root supergroup 0 2012-02-14 20:29
/user/root/examples/bin/work/reuters-out-seqdir
drwxr-xr-x - root supergroup 0 2012-02-14 20:32
/user/root/examples/bin/work/reuters-out-seqdir-sparse
[root@qxutest mahout-distribution-0.5]# hadoop fs -ls
/user/root/examples/bin/work/clusters
Found 1 items
-rw-r--r-- 2 root supergroup 139 2012-02-14 20:55
/user/root/examples/bin/work/clusters/part-randomSeed
[root@qxutest mahout-distribution-0.5]# ./bin/mahout kmeans -i
/user/root/examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors/ -c
/user/root/examples/bin/work/clusters -o
/user/root/examples/bin/work/reuters-kmeans -x 10 -ow
Running on hadoop, using HADOOP_HOME=/data/hadoop_cluster/hadoop-0.20.2/
HADOOP_CONF_DIR=/data/hadoop_cluster/hadoop-0.20.2/conf/
12/02/15 10:32:25 INFO common.AbstractJob: Command line arguments:
{--clusters=/user/root/examples/bin/work/clusters, --convergenceDelta=0.5,
--distanceMeasure=org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure,
--endPhase=2147483647,
--input=/user/root/examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors/,
--maxIter=10, --method=mapreduce,
--output=/user/root/examples/bin/work/reuters-kmeans, --overwrite=null,
--startPhase=0, --tempDir=temp}
12/02/15 10:32:25 INFO common.HadoopUtil: Deleting
/user/root/examples/bin/work/reuters-kmeans
12/02/15 10:32:25 INFO kmeans.KMeansDriver: Input:
/user/root/examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors Clusters
In: /user/root/examples/bin/work/clusters Out:
/user/root/examples/bin/work/reuters-kmeans Distance:
org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure
12/02/15 10:32:25 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10
num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}
12/02/15 10:32:25 INFO kmeans.KMeansDriver: K-Means Iteration 1
12/02/15 10:32:26 INFO input.FileInputFormat: Total input paths to process : 1
12/02/15 10:32:27 INFO mapred.JobClient: Running job: job_201202131515_0123
12/02/15 10:32:28 INFO mapred.JobClient: map 0% reduce 0%
12/02/15 10:32:38 INFO mapred.JobClient: Task Id :
attempt_201202131515_0123_m_000000_0, Status : FAILED
java.lang.IllegalStateException: No clusters found. Check your -c path.
at
org.apache.mahout.clustering.kmeans.KMeansMapper.setup(KMeansMapper.java:60)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Kmeans clustering error
> -----------------------
>
> Key: MAHOUT-504
> URL: https://issues.apache.org/jira/browse/MAHOUT-504
> Project: Mahout
> Issue Type: Bug
> Reporter: Zhen Guo
> Assignee: Robin Anil
> Fix For: 0.4
>
>
> I tried the Kmeans algorithm on the Synthetic Control data. The following
> error appears. I tried the Canopy algorithm, it is fine. This error is from
> Mapper. I am using Trunk.
> 10/09/20 19:40:06 INFO mapred.JobClient: Task Id :
> attempt_201008261432_1324_m_000000_0, Status : FAILED
> java.lang.IllegalStateException: Cluster is empty!
> at
> org.apache.mahout.clustering.kmeans.KMeansClusterMapper.setup(KMeansClusterMapper.java:57)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
