Sorry, I just noticed that the addition of sparse matrices is not implemented yet. I'll fix it soon.
Currently, an addition of dense matrices will only be run. # bin/hama examples rand 100 100 100% testA # bin/hama examples rand 100 100 100% testB # bin/hama examples add testA testB On Tue, Feb 16, 2010 at 11:07 PM, Edward J. Yoon <[email protected]> wrote: > Hi, Andrew > >>> java.lang.IndexOutOfBoundsException: v1.size != v2.size (25 != 23) > > Eh, It seems an un-expected exception. Let me check it out. > >>> Another question is about lack of "breadth first search" sample, >>> is this a sub-project of next release? > > Using M/R iterations, you can simply implement it -- > http://blog.udanax.org/2009/02/breadth-first-search-mapreduce.html > > But, an M/R iterative job is *awfully* slow, so I think it's not worth > to implement. > > Instead, we trying to implement a BSP-based graph computing framework > to improve the performance of graph algorithms on Hadoop. > > On Tue, Feb 16, 2010 at 10:41 PM, Andrew Milkowski > <[email protected]> wrote: >> Hi there >> >> Have following working setup: >> >> hadoop 0.20.1 >> hbase 0.20.3 >> hama latest SVN trunk version >> >> Notes: >> >> had to replace and rebuild hama with new hbase lib (upgraded to version >> 0.20.3) >> >> 1. successfully rand hama example to generate random matrix >> >> $HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixA >> >> Note: verified hbase table was generated (via shell list and scan) >> >> 2. moved on to other examples, generated 2 random matrices: >> >> $HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixA >> $HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixB >> >> then an attempt at distributed addition: >> >> $HAMA_HOME/bin/hama examples add1 matrixA matrixB >> >> >> 3. in similar venue tried multi, norms and and similarity (all generated >> exceptions) >> >> Exceptions appear to be hama code related (and not environmentally specific) >> >> Culprit could be in improper usage of input matrix data to add,multi, norms >> and similarity. >> >> Another question is about lack of "breadth first search" sample, is this a >> sub-project of next release? >> >> thanks >> >> below attached console stack trace of bash-3.2$ ./run_examples.sh (Note: >> ignore INFO zookeeper.ClientCnxn: Exception while closing send thread for >> session... exception that is a known "INFO" level exception present in >> zookeeper client/server socket code fragment) >> >> >> -------------------------------------------- >> >> run_examples.sh: >> >> #!/bin/bash >> >> export HAMA_JAR="${HAMA_HOME}/hama-0.2.0-dev-examples.jar" >> >> # cygwin path translation >> #if $cygwin; then >> # HAMA_CLASSPATH=`cygpath -p -w "$HAMA_CLASSPATH"` >> #fi >> >> $HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixA >> >> $HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixB >> >> $HAMA_HOME/bin/hama examples add matrixA matrixB >> >> exit $? >> >> >> >> >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:zookeeper.version=3.2.1-808558, built on 08/27/2009 18:48 GMT >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client environment:host.name >> =192.168.0.11 >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:java.version=1.6.0_17 >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:java.vendor=Apple Inc. >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:java.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:java.class.path=/opt/local/src/hama/current/bin/../conf:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/tools.jar:/opt/local/src/hama/current/bin/../build/classes:/opt/local/src/hama/current/bin/../build/test:/opt/local/src/hama/current/bin/../build:/opt/local/src/hama/current/bin/../hama-0.2.0-dev-examples.jar:/opt/local/src/hama/current/bin/../hama-0.2.0-dev.jar:/opt/local/src/hama/current/bin/../lib/AgileJSON-2009-03-30.jar:/opt/local/src/hama/current/bin/../lib/commons-cli-2.0-SNAPSHOT.jar:/opt/local/src/hama/current/bin/../lib/commons-el-from-jetty-5.1.4.jar:/opt/local/src/hama/current/bin/../lib/commons-httpclient-3.0.1.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-api-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-math-1.1.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-core.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3.jar:/opt/local/src/hama/current/bin/../lib/jasper-compiler-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/jasper-runtime-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/javacc.jar:/opt/local/src/hama/current/bin/../lib/jetty-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jetty-util-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jruby-complete-1.2.0.jar:/opt/local/src/hama/current/bin/../lib/json.jar:/opt/local/src/hama/current/bin/../lib/junit-3.8.1.jar:/opt/local/src/hama/current/bin/../lib/libthrift-r771587.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.13.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.15.jar:/opt/local/src/hama/current/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/xmlenc-0.52.jar:/opt/local/src/hama/current/bin/../lib/zookeeper-3.2.1.jar:/opt/local/src/hama/current/bin/../lib/jetty-ext/*.jar:/opt/local/src/hama/current/bin/../lib/findbugs/annotations.jar:/opt/local/src/hama/current/bin/../lib/findbugs/ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-analysis-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-commons-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-tree-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-util-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-xml-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/bcel.jar:/opt/local/src/hama/current/bin/../lib/findbugs/dom4j-full.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs-ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugsGUI.jar:/opt/local/src/hama/current/bin/../lib/findbugs/jsr305.jar:/opt/local/src/hama/current/bin/../lib/findbugs/plugin/coreplugin.jar:/opt/local/src/hadoop/current/conf:/opt/local/src/hadoop/current/conf >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:java.io.tmpdir=/var/folders/pM/pMTPzO70Hc4Gd2FB24T-dU+++TM/-Tmp-/ >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:java.compiler=<NA> >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client environment:os.name=Mac >> OS X >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:os.arch=x86_64 >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:os.version=10.6.2 >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client environment:user.name >> =hadoop >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:user.home=/Users/hadoop >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client >> environment:user.dir=/opt/local/hadoop/jobs/hama >> 10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Initiating client connection, >> connectString=localhost:2181 sessionTimeout=60000 >> watcher=org.apache.hadoop.hbase.client.hconnectionmanager$clientzkwatc...@2efb56b1 >> 10/02/16 08:28:29 INFO zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset >> is false >> 10/02/16 08:28:29 INFO zookeeper.ClientCnxn: Attempting connection to server >> localhost/127.0.0.1:2181 >> 10/02/16 08:28:29 INFO zookeeper.ClientCnxn: Priming connection to >> java.nio.channels.SocketChannel[connected >> local=/127.0.0.1:58238remote=localhost/ >> 127.0.0.1:2181] >> 10/02/16 08:28:29 INFO zookeeper.ClientCnxn: Server connection successful >> 10/02/16 08:28:30 INFO matrix.AbstractMatrix: Initializing the matrix >> storage. >> 10/02/16 08:28:32 INFO matrix.AbstractMatrix: Create Matrix >> SparseMatrix_randliqns >> 10/02/16 08:28:32 INFO matrix.AbstractMatrix: Create the 100 * 100 random >> matrix : SparseMatrix_randliqns >> Wrote input for Map #0 >> Wrote input for Map #1 >> Wrote input for Map #2 >> Wrote input for Map #3 >> Wrote input for Map #4 >> Wrote input for Map #5 >> Wrote input for Map #6 >> Wrote input for Map #7 >> Wrote input for Map #8 >> Wrote input for Map #9 >> 10/02/16 08:28:33 WARN mapred.JobClient: Use GenericOptionsParser for >> parsing the arguments. Applications should implement Tool for the same. >> 10/02/16 08:28:33 WARN mapred.JobClient: No job jar file set. User classes >> may not be found. See JobConf(Class) or JobConf#setJar(String). >> 10/02/16 08:28:33 INFO input.FileInputFormat: Total input paths to process : >> 10 >> 10/02/16 08:28:34 INFO mapred.JobClient: Running job: job_201002160823_0001 >> 10/02/16 08:28:35 INFO mapred.JobClient: map 0% reduce 0% >> 10/02/16 08:28:51 INFO mapred.JobClient: map 20% reduce 0% >> 10/02/16 08:29:00 INFO mapred.JobClient: map 40% reduce 0% >> 10/02/16 08:29:06 INFO mapred.JobClient: map 40% reduce 1% >> 10/02/16 08:29:09 INFO mapred.JobClient: map 60% reduce 2% >> 10/02/16 08:29:15 INFO mapred.JobClient: map 80% reduce 2% >> 10/02/16 08:29:18 INFO mapred.JobClient: map 80% reduce 3% >> 10/02/16 08:29:21 INFO mapred.JobClient: map 100% reduce 4% >> 10/02/16 08:29:24 INFO mapred.JobClient: map 100% reduce 5% >> 10/02/16 08:29:27 INFO mapred.JobClient: map 100% reduce 12% >> 10/02/16 08:29:30 INFO mapred.JobClient: map 100% reduce 20% >> 10/02/16 08:29:36 INFO mapred.JobClient: map 100% reduce 23% >> 10/02/16 08:29:39 INFO mapred.JobClient: map 100% reduce 33% >> 10/02/16 08:29:42 INFO mapred.JobClient: map 100% reduce 40% >> 10/02/16 08:29:48 INFO mapred.JobClient: map 100% reduce 43% >> 10/02/16 08:29:51 INFO mapred.JobClient: map 100% reduce 60% >> 10/02/16 08:30:01 INFO mapred.JobClient: map 100% reduce 70% >> 10/02/16 08:30:04 INFO mapred.JobClient: map 100% reduce 73% >> 10/02/16 08:30:07 INFO mapred.JobClient: map 100% reduce 80% >> 10/02/16 08:30:12 INFO mapred.JobClient: map 100% reduce 100% >> 10/02/16 08:30:14 INFO mapred.JobClient: Job complete: job_201002160823_0001 >> 10/02/16 08:30:14 INFO mapred.JobClient: Counters: 16 >> 10/02/16 08:30:14 INFO mapred.JobClient: Job Counters >> 10/02/16 08:30:14 INFO mapred.JobClient: Launched reduce tasks=10 >> 10/02/16 08:30:14 INFO mapred.JobClient: Launched map tasks=10 >> 10/02/16 08:30:14 INFO mapred.JobClient: Data-local map tasks=10 >> 10/02/16 08:30:14 INFO mapred.JobClient: FileSystemCounters >> 10/02/16 08:30:14 INFO mapred.JobClient: FILE_BYTES_READ=42372 >> 10/02/16 08:30:14 INFO mapred.JobClient: HDFS_BYTES_READ=1080 >> 10/02/16 08:30:14 INFO mapred.JobClient: FILE_BYTES_WRITTEN=87764 >> 10/02/16 08:30:14 INFO mapred.JobClient: Map-Reduce Framework >> 10/02/16 08:30:14 INFO mapred.JobClient: Reduce input groups=100 >> 10/02/16 08:30:14 INFO mapred.JobClient: Combine output records=0 >> 10/02/16 08:30:14 INFO mapred.JobClient: Map input records=10 >> 10/02/16 08:30:14 INFO mapred.JobClient: Reduce shuffle bytes=42463 >> 10/02/16 08:30:14 INFO mapred.JobClient: Reduce output records=100 >> 10/02/16 08:30:14 INFO mapred.JobClient: Spilled Records=200 >> 10/02/16 08:30:14 INFO mapred.JobClient: Map output bytes=41912 >> 10/02/16 08:30:14 INFO mapred.JobClient: Combine input records=0 >> 10/02/16 08:30:14 INFO mapred.JobClient: Map output records=100 >> 10/02/16 08:30:14 INFO mapred.JobClient: Reduce input records=100 >> 10/02/16 08:30:14 INFO zookeeper.ZooKeeper: Closing session: >> 0x126d6f5b8a90003 >> 10/02/16 08:30:14 INFO zookeeper.ClientCnxn: Closing ClientCnxn for session: >> 0x126d6f5b8a90003 >> 10/02/16 08:30:14 INFO zookeeper.ClientCnxn: Exception while closing send >> thread for session 0x126d6f5b8a90003 : Read error rc = -1 >> java.nio.DirectByteBuffer[pos=0 lim=4 cap=4] >> 10/02/16 08:30:14 WARN zookeeper.ClientCnxn: Ignoring exception during >> shutdown input >> java.net.SocketException: Socket is not connected >> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method) >> at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640) >> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360) >> at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999) >> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984) >> 10/02/16 08:30:14 INFO zookeeper.ClientCnxn: Disconnecting ClientCnxn for >> session: 0x126d6f5b8a90003 >> 10/02/16 08:30:14 INFO zookeeper.ZooKeeper: Session: 0x126d6f5b8a90003 >> closed >> 10/02/16 08:30:14 INFO zookeeper.ClientCnxn: EventThread shut down >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:zookeeper.version=3.2.1-808558, built on 08/27/2009 18:48 GMT >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client environment:host.name >> =192.168.0.11 >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:java.version=1.6.0_17 >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:java.vendor=Apple Inc. >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:java.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:java.class.path=/opt/local/src/hama/current/bin/../conf:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/tools.jar:/opt/local/src/hama/current/bin/../build/classes:/opt/local/src/hama/current/bin/../build/test:/opt/local/src/hama/current/bin/../build:/opt/local/src/hama/current/bin/../hama-0.2.0-dev-examples.jar:/opt/local/src/hama/current/bin/../hama-0.2.0-dev.jar:/opt/local/src/hama/current/bin/../lib/AgileJSON-2009-03-30.jar:/opt/local/src/hama/current/bin/../lib/commons-cli-2.0-SNAPSHOT.jar:/opt/local/src/hama/current/bin/../lib/commons-el-from-jetty-5.1.4.jar:/opt/local/src/hama/current/bin/../lib/commons-httpclient-3.0.1.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-api-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-math-1.1.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-core.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3.jar:/opt/local/src/hama/current/bin/../lib/jasper-compiler-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/jasper-runtime-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/javacc.jar:/opt/local/src/hama/current/bin/../lib/jetty-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jetty-util-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jruby-complete-1.2.0.jar:/opt/local/src/hama/current/bin/../lib/json.jar:/opt/local/src/hama/current/bin/../lib/junit-3.8.1.jar:/opt/local/src/hama/current/bin/../lib/libthrift-r771587.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.13.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.15.jar:/opt/local/src/hama/current/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/xmlenc-0.52.jar:/opt/local/src/hama/current/bin/../lib/zookeeper-3.2.1.jar:/opt/local/src/hama/current/bin/../lib/jetty-ext/*.jar:/opt/local/src/hama/current/bin/../lib/findbugs/annotations.jar:/opt/local/src/hama/current/bin/../lib/findbugs/ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-analysis-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-commons-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-tree-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-util-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-xml-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/bcel.jar:/opt/local/src/hama/current/bin/../lib/findbugs/dom4j-full.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs-ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugsGUI.jar:/opt/local/src/hama/current/bin/../lib/findbugs/jsr305.jar:/opt/local/src/hama/current/bin/../lib/findbugs/plugin/coreplugin.jar:/opt/local/src/hadoop/current/conf:/opt/local/src/hadoop/current/conf >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:java.io.tmpdir=/var/folders/pM/pMTPzO70Hc4Gd2FB24T-dU+++TM/-Tmp-/ >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:java.compiler=<NA> >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client environment:os.name=Mac >> OS X >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:os.arch=x86_64 >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:os.version=10.6.2 >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client environment:user.name >> =hadoop >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:user.home=/Users/hadoop >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client >> environment:user.dir=/opt/local/hadoop/jobs/hama >> 10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Initiating client connection, >> connectString=localhost:2181 sessionTimeout=60000 >> watcher=org.apache.hadoop.hbase.client.hconnectionmanager$clientzkwatc...@2efb56b1 >> 10/02/16 08:30:16 INFO zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset >> is false >> 10/02/16 08:30:16 INFO zookeeper.ClientCnxn: Attempting connection to server >> localhost/fe80:0:0:0:0:0:0:1%1:2181 >> 10/02/16 08:30:16 INFO zookeeper.ClientCnxn: Priming connection to >> java.nio.channels.SocketChannel[connected local=/fe80:0:0:0:0:0:0:1%1:58457 >> remote=localhost/fe80:0:0:0:0:0:0:1%1:2181] >> 10/02/16 08:30:16 INFO zookeeper.ClientCnxn: Server connection successful >> 10/02/16 08:30:17 INFO matrix.AbstractMatrix: Initializing the matrix >> storage. >> 10/02/16 08:30:19 INFO matrix.AbstractMatrix: Create Matrix >> SparseMatrix_randflbeh >> 10/02/16 08:30:19 INFO matrix.AbstractMatrix: Create the 100 * 100 random >> matrix : SparseMatrix_randflbeh >> Wrote input for Map #0 >> Wrote input for Map #1 >> Wrote input for Map #2 >> Wrote input for Map #3 >> Wrote input for Map #4 >> Wrote input for Map #5 >> Wrote input for Map #6 >> Wrote input for Map #7 >> Wrote input for Map #8 >> Wrote input for Map #9 >> 10/02/16 08:30:20 WARN mapred.JobClient: Use GenericOptionsParser for >> parsing the arguments. Applications should implement Tool for the same. >> 10/02/16 08:30:20 WARN mapred.JobClient: No job jar file set. User classes >> may not be found. See JobConf(Class) or JobConf#setJar(String). >> 10/02/16 08:30:20 INFO input.FileInputFormat: Total input paths to process : >> 10 >> 10/02/16 08:30:21 INFO mapred.JobClient: Running job: job_201002160823_0002 >> 10/02/16 08:30:22 INFO mapred.JobClient: map 0% reduce 0% >> 10/02/16 08:30:33 INFO mapred.JobClient: map 20% reduce 0% >> 10/02/16 08:30:42 INFO mapred.JobClient: map 40% reduce 0% >> 10/02/16 08:30:45 INFO mapred.JobClient: map 40% reduce 1% >> 10/02/16 08:30:51 INFO mapred.JobClient: map 60% reduce 2% >> 10/02/16 08:30:57 INFO mapred.JobClient: map 80% reduce 2% >> 10/02/16 08:31:00 INFO mapred.JobClient: map 80% reduce 4% >> 10/02/16 08:31:03 INFO mapred.JobClient: map 100% reduce 4% >> 10/02/16 08:31:06 INFO mapred.JobClient: map 100% reduce 12% >> 10/02/16 08:31:12 INFO mapred.JobClient: map 100% reduce 30% >> 10/02/16 08:31:22 INFO mapred.JobClient: map 100% reduce 43% >> 10/02/16 08:31:25 INFO mapred.JobClient: map 100% reduce 50% >> 10/02/16 08:31:31 INFO mapred.JobClient: map 100% reduce 53% >> 10/02/16 08:31:34 INFO mapred.JobClient: map 100% reduce 63% >> 10/02/16 08:31:37 INFO mapred.JobClient: map 100% reduce 70% >> 10/02/16 08:31:43 INFO mapred.JobClient: map 100% reduce 73% >> 10/02/16 08:31:46 INFO mapred.JobClient: map 100% reduce 83% >> 10/02/16 08:31:49 INFO mapred.JobClient: map 100% reduce 90% >> 10/02/16 08:31:55 INFO mapred.JobClient: map 100% reduce 93% >> 10/02/16 08:31:58 INFO mapred.JobClient: map 100% reduce 100% >> 10/02/16 08:32:00 INFO mapred.JobClient: Job complete: job_201002160823_0002 >> 10/02/16 08:32:00 INFO mapred.JobClient: Counters: 16 >> 10/02/16 08:32:00 INFO mapred.JobClient: Job Counters >> 10/02/16 08:32:00 INFO mapred.JobClient: Launched reduce tasks=10 >> 10/02/16 08:32:00 INFO mapred.JobClient: Launched map tasks=10 >> 10/02/16 08:32:00 INFO mapred.JobClient: Data-local map tasks=10 >> 10/02/16 08:32:00 INFO mapred.JobClient: FileSystemCounters >> 10/02/16 08:32:00 INFO mapred.JobClient: FILE_BYTES_READ=42512 >> 10/02/16 08:32:00 INFO mapred.JobClient: HDFS_BYTES_READ=1080 >> 10/02/16 08:32:00 INFO mapred.JobClient: FILE_BYTES_WRITTEN=88044 >> 10/02/16 08:32:00 INFO mapred.JobClient: Map-Reduce Framework >> 10/02/16 08:32:00 INFO mapred.JobClient: Reduce input groups=100 >> 10/02/16 08:32:00 INFO mapred.JobClient: Combine output records=0 >> 10/02/16 08:32:00 INFO mapred.JobClient: Map input records=10 >> 10/02/16 08:32:00 INFO mapred.JobClient: Reduce shuffle bytes=42603 >> 10/02/16 08:32:00 INFO mapred.JobClient: Reduce output records=100 >> 10/02/16 08:32:00 INFO mapred.JobClient: Spilled Records=200 >> 10/02/16 08:32:00 INFO mapred.JobClient: Map output bytes=42052 >> 10/02/16 08:32:00 INFO mapred.JobClient: Combine input records=0 >> 10/02/16 08:32:00 INFO mapred.JobClient: Map output records=100 >> 10/02/16 08:32:00 INFO mapred.JobClient: Reduce input records=100 >> 10/02/16 08:32:00 INFO zookeeper.ZooKeeper: Closing session: >> 0x126d6f5b8a9000e >> 10/02/16 08:32:00 INFO zookeeper.ClientCnxn: Closing ClientCnxn for session: >> 0x126d6f5b8a9000e >> 10/02/16 08:32:00 INFO zookeeper.ClientCnxn: Exception while closing send >> thread for session 0x126d6f5b8a9000e : Read error rc = -1 >> java.nio.DirectByteBuffer[pos=0 lim=4 cap=4] >> 10/02/16 08:32:00 WARN zookeeper.ClientCnxn: Ignoring exception during >> shutdown input >> java.net.SocketException: Socket is not connected >> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method) >> at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640) >> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360) >> at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999) >> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984) >> 10/02/16 08:32:00 INFO zookeeper.ClientCnxn: Disconnecting ClientCnxn for >> session: 0x126d6f5b8a9000e >> 10/02/16 08:32:00 INFO zookeeper.ZooKeeper: Session: 0x126d6f5b8a9000e >> closed >> 10/02/16 08:32:00 INFO zookeeper.ClientCnxn: EventThread shut down >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:zookeeper.version=3.2.1-808558, built on 08/27/2009 18:48 GMT >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client environment:host.name >> =192.168.0.11 >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:java.version=1.6.0_17 >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:java.vendor=Apple Inc. >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:java.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:java.class.path=/opt/local/src/hama/current/bin/../conf:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/tools.jar:/opt/local/src/hama/current/bin/../build/classes:/opt/local/src/hama/current/bin/../build/test:/opt/local/src/hama/current/bin/../build:/opt/local/src/hama/current/bin/../hama-0.2.0-dev-examples.jar:/opt/local/src/hama/current/bin/../hama-0.2.0-dev.jar:/opt/local/src/hama/current/bin/../lib/AgileJSON-2009-03-30.jar:/opt/local/src/hama/current/bin/../lib/commons-cli-2.0-SNAPSHOT.jar:/opt/local/src/hama/current/bin/../lib/commons-el-from-jetty-5.1.4.jar:/opt/local/src/hama/current/bin/../lib/commons-httpclient-3.0.1.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-api-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-math-1.1.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-core.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3.jar:/opt/local/src/hama/current/bin/../lib/jasper-compiler-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/jasper-runtime-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/javacc.jar:/opt/local/src/hama/current/bin/../lib/jetty-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jetty-util-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jruby-complete-1.2.0.jar:/opt/local/src/hama/current/bin/../lib/json.jar:/opt/local/src/hama/current/bin/../lib/junit-3.8.1.jar:/opt/local/src/hama/current/bin/../lib/libthrift-r771587.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.13.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.15.jar:/opt/local/src/hama/current/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/xmlenc-0.52.jar:/opt/local/src/hama/current/bin/../lib/zookeeper-3.2.1.jar:/opt/local/src/hama/current/bin/../lib/jetty-ext/*.jar:/opt/local/src/hama/current/bin/../lib/findbugs/annotations.jar:/opt/local/src/hama/current/bin/../lib/findbugs/ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-analysis-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-commons-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-tree-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-util-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-xml-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/bcel.jar:/opt/local/src/hama/current/bin/../lib/findbugs/dom4j-full.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs-ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugsGUI.jar:/opt/local/src/hama/current/bin/../lib/findbugs/jsr305.jar:/opt/local/src/hama/current/bin/../lib/findbugs/plugin/coreplugin.jar:/opt/local/src/hadoop/current/conf:/opt/local/src/hadoop/current/conf >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:java.io.tmpdir=/var/folders/pM/pMTPzO70Hc4Gd2FB24T-dU+++TM/-Tmp-/ >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:java.compiler=<NA> >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client environment:os.name=Mac >> OS X >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:os.arch=x86_64 >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:os.version=10.6.2 >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client environment:user.name >> =hadoop >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:user.home=/Users/hadoop >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client >> environment:user.dir=/opt/local/hadoop/jobs/hama >> 10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Initiating client connection, >> connectString=localhost:2181 sessionTimeout=60000 >> watcher=org.apache.hadoop.hbase.client.hconnectionmanager$clientzkwatc...@76f8968f >> 10/02/16 08:32:02 INFO zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset >> is false >> 10/02/16 08:32:02 INFO zookeeper.ClientCnxn: Attempting connection to server >> localhost/0:0:0:0:0:0:0:1:2181 >> 10/02/16 08:32:02 INFO zookeeper.ClientCnxn: Priming connection to >> java.nio.channels.SocketChannel[connected local=/0:0:0:0:0:0:0:1%0:58679 >> remote=localhost/0:0:0:0:0:0:0:1:2181] >> 10/02/16 08:32:02 INFO zookeeper.ClientCnxn: Server connection successful >> C = A + B >> 10/02/16 08:32:05 INFO matrix.AbstractMatrix: Initializing the matrix >> storage. >> 10/02/16 08:32:07 INFO matrix.AbstractMatrix: Create Matrix >> DenseMatrix_randwhqtl >> 10/02/16 08:32:07 WARN mapred.JobClient: Use GenericOptionsParser for >> parsing the arguments. Applications should implement Tool for the same. >> 10/02/16 08:32:07 WARN mapred.JobClient: No job jar file set. User classes >> may not be found. See JobConf(Class) or JobConf#setJar(String). >> 10/02/16 08:32:08 INFO mapred.JobClient: Running job: job_201002160823_0003 >> 10/02/16 08:32:09 INFO mapred.JobClient: map 0% reduce 0% >> 10/02/16 08:32:23 INFO mapred.JobClient: Task Id : >> attempt_201002160823_0003_m_000000_0, Status : FAILED >> java.lang.IndexOutOfBoundsException: v1.size != v2.size (25 != 23) >> at >> org.apache.hama.matrix.AbstractVector.checkComformantSize(AbstractVector.java:92) >> at org.apache.hama.matrix.DenseVector.add(DenseVector.java:150) >> at >> org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:35) >> at >> org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:18) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) >> at org.apache.hadoop.mapred.Child.main(Child.java:170) >> >> 10/02/16 08:32:33 INFO mapred.JobClient: Task Id : >> attempt_201002160823_0003_m_000000_1, Status : FAILED >> java.lang.IndexOutOfBoundsException: v1.size != v2.size (25 != 23) >> at >> org.apache.hama.matrix.AbstractVector.checkComformantSize(AbstractVector.java:92) >> at org.apache.hama.matrix.DenseVector.add(DenseVector.java:150) >> at >> org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:35) >> at >> org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:18) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) >> at org.apache.hadoop.mapred.Child.main(Child.java:170) >> >> 10/02/16 08:32:42 INFO mapred.JobClient: Task Id : >> attempt_201002160823_0003_m_000000_2, Status : FAILED >> java.lang.IndexOutOfBoundsException: v1.size != v2.size (25 != 23) >> at >> org.apache.hama.matrix.AbstractVector.checkComformantSize(AbstractVector.java:92) >> at org.apache.hama.matrix.DenseVector.add(DenseVector.java:150) >> at >> org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:35) >> at >> org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:18) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) >> at org.apache.hadoop.mapred.Child.main(Child.java:170) >> >> 10/02/16 08:32:54 INFO mapred.JobClient: Job complete: job_201002160823_0003 >> 10/02/16 08:32:54 INFO mapred.JobClient: Counters: 3 >> 10/02/16 08:32:54 INFO mapred.JobClient: Job Counters >> 10/02/16 08:32:54 INFO mapred.JobClient: Launched map tasks=4 >> 10/02/16 08:32:54 INFO mapred.JobClient: Data-local map tasks=4 >> 10/02/16 08:32:54 INFO mapred.JobClient: Failed map tasks=1 >> 10/02/16 08:32:56 INFO client.HBaseAdmin: Disabled DenseMatrix_randwhqtl >> 10/02/16 08:32:56 INFO client.HBaseAdmin: Deleted DenseMatrix_randwhqtl >> 10/02/16 08:32:56 INFO zookeeper.ZooKeeper: Closing session: >> 0x126d6f5b8a90019 >> 10/02/16 08:32:56 INFO zookeeper.ClientCnxn: Closing ClientCnxn for session: >> 0x126d6f5b8a90019 >> 10/02/16 08:32:56 INFO zookeeper.ClientCnxn: Exception while closing send >> thread for session 0x126d6f5b8a90019 : Read error rc = -1 >> java.nio.DirectByteBuffer[pos=0 lim=4 cap=4] >> 10/02/16 08:32:56 WARN zookeeper.ClientCnxn: Ignoring exception during >> shutdown input >> java.net.SocketException: Socket is not connected >> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method) >> at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640) >> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360) >> at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999) >> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984) >> 10/02/16 08:32:56 INFO zookeeper.ClientCnxn: Disconnecting ClientCnxn for >> session: 0x126d6f5b8a90019 >> 10/02/16 08:32:56 INFO zookeeper.ZooKeeper: Session: 0x126d6f5b8a90019 >> closed >> 10/02/16 08:32:56 INFO zookeeper.ClientCnxn: EventThread shut down >> run (rand, rand, add) >> > > > > -- > Best Regards, Edward J. Yoon @ NHN, corp. > [email protected] > http://blog.udanax.org > -- Best Regards, Edward J. Yoon @ NHN, corp. [email protected] http://blog.udanax.org
