Hi there
Have following working setup:
hadoop 0.20.1
hbase 0.20.3
hama latest SVN trunk version
Notes:
had to replace and rebuild hama with new hbase lib (upgraded to version
0.20.3)
1. successfully rand hama example to generate random matrix
$HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixA
Note: verified hbase table was generated (via shell list and scan)
2. moved on to other examples, generated 2 random matrices:
$HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixA
$HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixB
then an attempt at distributed addition:
$HAMA_HOME/bin/hama examples add1 matrixA matrixB
3. in similar venue tried multi, norms and and similarity (all generated
exceptions)
Exceptions appear to be hama code related (and not environmentally specific)
Culprit could be in improper usage of input matrix data to add,multi, norms
and similarity.
Another question is about lack of "breadth first search" sample, is this a
sub-project of next release?
thanks
below attached console stack trace of bash-3.2$ ./run_examples.sh (Note:
ignore INFO zookeeper.ClientCnxn: Exception while closing send thread for
session... exception that is a known "INFO" level exception present in
zookeeper client/server socket code fragment)
--------------------------------------------
run_examples.sh:
#!/bin/bash
export HAMA_JAR="${HAMA_HOME}/hama-0.2.0-dev-examples.jar"
# cygwin path translation
#if $cygwin; then
# HAMA_CLASSPATH=`cygpath -p -w "$HAMA_CLASSPATH"`
#fi
$HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixA
$HAMA_HOME/bin/hama examples rand -m 10 -r 10 100 100 30.5% matrixB
$HAMA_HOME/bin/hama examples add matrixA matrixB
exit $?
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.1-808558, built on 08/27/2009 18:48 GMT
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client environment:host.name
=192.168.0.11
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_17
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Apple Inc.
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:java.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:java.class.path=/opt/local/src/hama/current/bin/../conf:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/tools.jar:/opt/local/src/hama/current/bin/../build/classes:/opt/local/src/hama/current/bin/../build/test:/opt/local/src/hama/current/bin/../build:/opt/local/src/hama/current/bin/../hama-0.2.0-dev-examples.jar:/opt/local/src/hama/current/bin/../hama-0.2.0-dev.jar:/opt/local/src/hama/current/bin/../lib/AgileJSON-2009-03-30.jar:/opt/local/src/hama/current/bin/../lib/commons-cli-2.0-SNAPSHOT.jar:/opt/local/src/hama/current/bin/../lib/commons-el-from-jetty-5.1.4.jar:/opt/local/src/hama/current/bin/../lib/commons-httpclient-3.0.1.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-api-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-math-1.1.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-core.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3.jar:/opt/local/src/hama/current/bin/../lib/jasper-compiler-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/jasper-runtime-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/javacc.jar:/opt/local/src/hama/current/bin/../lib/jetty-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jetty-util-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jruby-complete-1.2.0.jar:/opt/local/src/hama/current/bin/../lib/json.jar:/opt/local/src/hama/current/bin/../lib/junit-3.8.1.jar:/opt/local/src/hama/current/bin/../lib/libthrift-r771587.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.13.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.15.jar:/opt/local/src/hama/current/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/xmlenc-0.52.jar:/opt/local/src/hama/current/bin/../lib/zookeeper-3.2.1.jar:/opt/local/src/hama/current/bin/../lib/jetty-ext/*.jar:/opt/local/src/hama/current/bin/../lib/findbugs/annotations.jar:/opt/local/src/hama/current/bin/../lib/findbugs/ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-analysis-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-commons-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-tree-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-util-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-xml-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/bcel.jar:/opt/local/src/hama/current/bin/../lib/findbugs/dom4j-full.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs-ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugsGUI.jar:/opt/local/src/hama/current/bin/../lib/findbugs/jsr305.jar:/opt/local/src/hama/current/bin/../lib/findbugs/plugin/coreplugin.jar:/opt/local/src/hadoop/current/conf:/opt/local/src/hadoop/current/conf
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/var/folders/pM/pMTPzO70Hc4Gd2FB24T-dU+++TM/-Tmp-/
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client environment:os.name=Mac
OS X
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:os.arch=x86_64
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:os.version=10.6.2
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client environment:user.name
=hadoop
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:user.home=/Users/hadoop
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Client
environment:user.dir=/opt/local/hadoop/jobs/hama
10/02/16 08:28:29 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2181 sessionTimeout=60000
watcher=org.apache.hadoop.hbase.client.hconnectionmanager$clientzkwatc...@2efb56b1
10/02/16 08:28:29 INFO zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset
is false
10/02/16 08:28:29 INFO zookeeper.ClientCnxn: Attempting connection to server
localhost/127.0.0.1:2181
10/02/16 08:28:29 INFO zookeeper.ClientCnxn: Priming connection to
java.nio.channels.SocketChannel[connected
local=/127.0.0.1:58238remote=localhost/
127.0.0.1:2181]
10/02/16 08:28:29 INFO zookeeper.ClientCnxn: Server connection successful
10/02/16 08:28:30 INFO matrix.AbstractMatrix: Initializing the matrix
storage.
10/02/16 08:28:32 INFO matrix.AbstractMatrix: Create Matrix
SparseMatrix_randliqns
10/02/16 08:28:32 INFO matrix.AbstractMatrix: Create the 100 * 100 random
matrix : SparseMatrix_randliqns
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
10/02/16 08:28:33 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
10/02/16 08:28:33 WARN mapred.JobClient: No job jar file set. User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
10/02/16 08:28:33 INFO input.FileInputFormat: Total input paths to process :
10
10/02/16 08:28:34 INFO mapred.JobClient: Running job: job_201002160823_0001
10/02/16 08:28:35 INFO mapred.JobClient: map 0% reduce 0%
10/02/16 08:28:51 INFO mapred.JobClient: map 20% reduce 0%
10/02/16 08:29:00 INFO mapred.JobClient: map 40% reduce 0%
10/02/16 08:29:06 INFO mapred.JobClient: map 40% reduce 1%
10/02/16 08:29:09 INFO mapred.JobClient: map 60% reduce 2%
10/02/16 08:29:15 INFO mapred.JobClient: map 80% reduce 2%
10/02/16 08:29:18 INFO mapred.JobClient: map 80% reduce 3%
10/02/16 08:29:21 INFO mapred.JobClient: map 100% reduce 4%
10/02/16 08:29:24 INFO mapred.JobClient: map 100% reduce 5%
10/02/16 08:29:27 INFO mapred.JobClient: map 100% reduce 12%
10/02/16 08:29:30 INFO mapred.JobClient: map 100% reduce 20%
10/02/16 08:29:36 INFO mapred.JobClient: map 100% reduce 23%
10/02/16 08:29:39 INFO mapred.JobClient: map 100% reduce 33%
10/02/16 08:29:42 INFO mapred.JobClient: map 100% reduce 40%
10/02/16 08:29:48 INFO mapred.JobClient: map 100% reduce 43%
10/02/16 08:29:51 INFO mapred.JobClient: map 100% reduce 60%
10/02/16 08:30:01 INFO mapred.JobClient: map 100% reduce 70%
10/02/16 08:30:04 INFO mapred.JobClient: map 100% reduce 73%
10/02/16 08:30:07 INFO mapred.JobClient: map 100% reduce 80%
10/02/16 08:30:12 INFO mapred.JobClient: map 100% reduce 100%
10/02/16 08:30:14 INFO mapred.JobClient: Job complete: job_201002160823_0001
10/02/16 08:30:14 INFO mapred.JobClient: Counters: 16
10/02/16 08:30:14 INFO mapred.JobClient: Job Counters
10/02/16 08:30:14 INFO mapred.JobClient: Launched reduce tasks=10
10/02/16 08:30:14 INFO mapred.JobClient: Launched map tasks=10
10/02/16 08:30:14 INFO mapred.JobClient: Data-local map tasks=10
10/02/16 08:30:14 INFO mapred.JobClient: FileSystemCounters
10/02/16 08:30:14 INFO mapred.JobClient: FILE_BYTES_READ=42372
10/02/16 08:30:14 INFO mapred.JobClient: HDFS_BYTES_READ=1080
10/02/16 08:30:14 INFO mapred.JobClient: FILE_BYTES_WRITTEN=87764
10/02/16 08:30:14 INFO mapred.JobClient: Map-Reduce Framework
10/02/16 08:30:14 INFO mapred.JobClient: Reduce input groups=100
10/02/16 08:30:14 INFO mapred.JobClient: Combine output records=0
10/02/16 08:30:14 INFO mapred.JobClient: Map input records=10
10/02/16 08:30:14 INFO mapred.JobClient: Reduce shuffle bytes=42463
10/02/16 08:30:14 INFO mapred.JobClient: Reduce output records=100
10/02/16 08:30:14 INFO mapred.JobClient: Spilled Records=200
10/02/16 08:30:14 INFO mapred.JobClient: Map output bytes=41912
10/02/16 08:30:14 INFO mapred.JobClient: Combine input records=0
10/02/16 08:30:14 INFO mapred.JobClient: Map output records=100
10/02/16 08:30:14 INFO mapred.JobClient: Reduce input records=100
10/02/16 08:30:14 INFO zookeeper.ZooKeeper: Closing session:
0x126d6f5b8a90003
10/02/16 08:30:14 INFO zookeeper.ClientCnxn: Closing ClientCnxn for session:
0x126d6f5b8a90003
10/02/16 08:30:14 INFO zookeeper.ClientCnxn: Exception while closing send
thread for session 0x126d6f5b8a90003 : Read error rc = -1
java.nio.DirectByteBuffer[pos=0 lim=4 cap=4]
10/02/16 08:30:14 WARN zookeeper.ClientCnxn: Ignoring exception during
shutdown input
java.net.SocketException: Socket is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
10/02/16 08:30:14 INFO zookeeper.ClientCnxn: Disconnecting ClientCnxn for
session: 0x126d6f5b8a90003
10/02/16 08:30:14 INFO zookeeper.ZooKeeper: Session: 0x126d6f5b8a90003
closed
10/02/16 08:30:14 INFO zookeeper.ClientCnxn: EventThread shut down
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.1-808558, built on 08/27/2009 18:48 GMT
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client environment:host.name
=192.168.0.11
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_17
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Apple Inc.
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:java.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:java.class.path=/opt/local/src/hama/current/bin/../conf:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/tools.jar:/opt/local/src/hama/current/bin/../build/classes:/opt/local/src/hama/current/bin/../build/test:/opt/local/src/hama/current/bin/../build:/opt/local/src/hama/current/bin/../hama-0.2.0-dev-examples.jar:/opt/local/src/hama/current/bin/../hama-0.2.0-dev.jar:/opt/local/src/hama/current/bin/../lib/AgileJSON-2009-03-30.jar:/opt/local/src/hama/current/bin/../lib/commons-cli-2.0-SNAPSHOT.jar:/opt/local/src/hama/current/bin/../lib/commons-el-from-jetty-5.1.4.jar:/opt/local/src/hama/current/bin/../lib/commons-httpclient-3.0.1.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-api-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-math-1.1.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-core.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3.jar:/opt/local/src/hama/current/bin/../lib/jasper-compiler-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/jasper-runtime-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/javacc.jar:/opt/local/src/hama/current/bin/../lib/jetty-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jetty-util-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jruby-complete-1.2.0.jar:/opt/local/src/hama/current/bin/../lib/json.jar:/opt/local/src/hama/current/bin/../lib/junit-3.8.1.jar:/opt/local/src/hama/current/bin/../lib/libthrift-r771587.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.13.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.15.jar:/opt/local/src/hama/current/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/xmlenc-0.52.jar:/opt/local/src/hama/current/bin/../lib/zookeeper-3.2.1.jar:/opt/local/src/hama/current/bin/../lib/jetty-ext/*.jar:/opt/local/src/hama/current/bin/../lib/findbugs/annotations.jar:/opt/local/src/hama/current/bin/../lib/findbugs/ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-analysis-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-commons-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-tree-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-util-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-xml-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/bcel.jar:/opt/local/src/hama/current/bin/../lib/findbugs/dom4j-full.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs-ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugsGUI.jar:/opt/local/src/hama/current/bin/../lib/findbugs/jsr305.jar:/opt/local/src/hama/current/bin/../lib/findbugs/plugin/coreplugin.jar:/opt/local/src/hadoop/current/conf:/opt/local/src/hadoop/current/conf
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/var/folders/pM/pMTPzO70Hc4Gd2FB24T-dU+++TM/-Tmp-/
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client environment:os.name=Mac
OS X
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:os.arch=x86_64
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:os.version=10.6.2
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client environment:user.name
=hadoop
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:user.home=/Users/hadoop
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Client
environment:user.dir=/opt/local/hadoop/jobs/hama
10/02/16 08:30:16 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2181 sessionTimeout=60000
watcher=org.apache.hadoop.hbase.client.hconnectionmanager$clientzkwatc...@2efb56b1
10/02/16 08:30:16 INFO zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset
is false
10/02/16 08:30:16 INFO zookeeper.ClientCnxn: Attempting connection to server
localhost/fe80:0:0:0:0:0:0:1%1:2181
10/02/16 08:30:16 INFO zookeeper.ClientCnxn: Priming connection to
java.nio.channels.SocketChannel[connected local=/fe80:0:0:0:0:0:0:1%1:58457
remote=localhost/fe80:0:0:0:0:0:0:1%1:2181]
10/02/16 08:30:16 INFO zookeeper.ClientCnxn: Server connection successful
10/02/16 08:30:17 INFO matrix.AbstractMatrix: Initializing the matrix
storage.
10/02/16 08:30:19 INFO matrix.AbstractMatrix: Create Matrix
SparseMatrix_randflbeh
10/02/16 08:30:19 INFO matrix.AbstractMatrix: Create the 100 * 100 random
matrix : SparseMatrix_randflbeh
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
10/02/16 08:30:20 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
10/02/16 08:30:20 WARN mapred.JobClient: No job jar file set. User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
10/02/16 08:30:20 INFO input.FileInputFormat: Total input paths to process :
10
10/02/16 08:30:21 INFO mapred.JobClient: Running job: job_201002160823_0002
10/02/16 08:30:22 INFO mapred.JobClient: map 0% reduce 0%
10/02/16 08:30:33 INFO mapred.JobClient: map 20% reduce 0%
10/02/16 08:30:42 INFO mapred.JobClient: map 40% reduce 0%
10/02/16 08:30:45 INFO mapred.JobClient: map 40% reduce 1%
10/02/16 08:30:51 INFO mapred.JobClient: map 60% reduce 2%
10/02/16 08:30:57 INFO mapred.JobClient: map 80% reduce 2%
10/02/16 08:31:00 INFO mapred.JobClient: map 80% reduce 4%
10/02/16 08:31:03 INFO mapred.JobClient: map 100% reduce 4%
10/02/16 08:31:06 INFO mapred.JobClient: map 100% reduce 12%
10/02/16 08:31:12 INFO mapred.JobClient: map 100% reduce 30%
10/02/16 08:31:22 INFO mapred.JobClient: map 100% reduce 43%
10/02/16 08:31:25 INFO mapred.JobClient: map 100% reduce 50%
10/02/16 08:31:31 INFO mapred.JobClient: map 100% reduce 53%
10/02/16 08:31:34 INFO mapred.JobClient: map 100% reduce 63%
10/02/16 08:31:37 INFO mapred.JobClient: map 100% reduce 70%
10/02/16 08:31:43 INFO mapred.JobClient: map 100% reduce 73%
10/02/16 08:31:46 INFO mapred.JobClient: map 100% reduce 83%
10/02/16 08:31:49 INFO mapred.JobClient: map 100% reduce 90%
10/02/16 08:31:55 INFO mapred.JobClient: map 100% reduce 93%
10/02/16 08:31:58 INFO mapred.JobClient: map 100% reduce 100%
10/02/16 08:32:00 INFO mapred.JobClient: Job complete: job_201002160823_0002
10/02/16 08:32:00 INFO mapred.JobClient: Counters: 16
10/02/16 08:32:00 INFO mapred.JobClient: Job Counters
10/02/16 08:32:00 INFO mapred.JobClient: Launched reduce tasks=10
10/02/16 08:32:00 INFO mapred.JobClient: Launched map tasks=10
10/02/16 08:32:00 INFO mapred.JobClient: Data-local map tasks=10
10/02/16 08:32:00 INFO mapred.JobClient: FileSystemCounters
10/02/16 08:32:00 INFO mapred.JobClient: FILE_BYTES_READ=42512
10/02/16 08:32:00 INFO mapred.JobClient: HDFS_BYTES_READ=1080
10/02/16 08:32:00 INFO mapred.JobClient: FILE_BYTES_WRITTEN=88044
10/02/16 08:32:00 INFO mapred.JobClient: Map-Reduce Framework
10/02/16 08:32:00 INFO mapred.JobClient: Reduce input groups=100
10/02/16 08:32:00 INFO mapred.JobClient: Combine output records=0
10/02/16 08:32:00 INFO mapred.JobClient: Map input records=10
10/02/16 08:32:00 INFO mapred.JobClient: Reduce shuffle bytes=42603
10/02/16 08:32:00 INFO mapred.JobClient: Reduce output records=100
10/02/16 08:32:00 INFO mapred.JobClient: Spilled Records=200
10/02/16 08:32:00 INFO mapred.JobClient: Map output bytes=42052
10/02/16 08:32:00 INFO mapred.JobClient: Combine input records=0
10/02/16 08:32:00 INFO mapred.JobClient: Map output records=100
10/02/16 08:32:00 INFO mapred.JobClient: Reduce input records=100
10/02/16 08:32:00 INFO zookeeper.ZooKeeper: Closing session:
0x126d6f5b8a9000e
10/02/16 08:32:00 INFO zookeeper.ClientCnxn: Closing ClientCnxn for session:
0x126d6f5b8a9000e
10/02/16 08:32:00 INFO zookeeper.ClientCnxn: Exception while closing send
thread for session 0x126d6f5b8a9000e : Read error rc = -1
java.nio.DirectByteBuffer[pos=0 lim=4 cap=4]
10/02/16 08:32:00 WARN zookeeper.ClientCnxn: Ignoring exception during
shutdown input
java.net.SocketException: Socket is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
10/02/16 08:32:00 INFO zookeeper.ClientCnxn: Disconnecting ClientCnxn for
session: 0x126d6f5b8a9000e
10/02/16 08:32:00 INFO zookeeper.ZooKeeper: Session: 0x126d6f5b8a9000e
closed
10/02/16 08:32:00 INFO zookeeper.ClientCnxn: EventThread shut down
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.1-808558, built on 08/27/2009 18:48 GMT
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client environment:host.name
=192.168.0.11
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_17
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Apple Inc.
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:java.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:java.class.path=/opt/local/src/hama/current/bin/../conf:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/tools.jar:/opt/local/src/hama/current/bin/../build/classes:/opt/local/src/hama/current/bin/../build/test:/opt/local/src/hama/current/bin/../build:/opt/local/src/hama/current/bin/../hama-0.2.0-dev-examples.jar:/opt/local/src/hama/current/bin/../hama-0.2.0-dev.jar:/opt/local/src/hama/current/bin/../lib/AgileJSON-2009-03-30.jar:/opt/local/src/hama/current/bin/../lib/commons-cli-2.0-SNAPSHOT.jar:/opt/local/src/hama/current/bin/../lib/commons-el-from-jetty-5.1.4.jar:/opt/local/src/hama/current/bin/../lib/commons-httpclient-3.0.1.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-logging-api-1.0.4.jar:/opt/local/src/hama/current/bin/../lib/commons-math-1.1.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-core.jar:/opt/local/src/hama/current/bin/../lib/hadoop-0.20.1-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3-test.jar:/opt/local/src/hama/current/bin/../lib/hbase-0.20.3.jar:/opt/local/src/hama/current/bin/../lib/jasper-compiler-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/jasper-runtime-5.5.12.jar:/opt/local/src/hama/current/bin/../lib/javacc.jar:/opt/local/src/hama/current/bin/../lib/jetty-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jetty-util-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/jruby-complete-1.2.0.jar:/opt/local/src/hama/current/bin/../lib/json.jar:/opt/local/src/hama/current/bin/../lib/junit-3.8.1.jar:/opt/local/src/hama/current/bin/../lib/libthrift-r771587.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.13.jar:/opt/local/src/hama/current/bin/../lib/log4j-1.2.15.jar:/opt/local/src/hama/current/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/local/src/hama/current/bin/../lib/xmlenc-0.52.jar:/opt/local/src/hama/current/bin/../lib/zookeeper-3.2.1.jar:/opt/local/src/hama/current/bin/../lib/jetty-ext/*.jar:/opt/local/src/hama/current/bin/../lib/findbugs/annotations.jar:/opt/local/src/hama/current/bin/../lib/findbugs/ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-analysis-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-commons-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-tree-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-util-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/asm-xml-3.0.jar:/opt/local/src/hama/current/bin/../lib/findbugs/bcel.jar:/opt/local/src/hama/current/bin/../lib/findbugs/dom4j-full.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs-ant.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugs.jar:/opt/local/src/hama/current/bin/../lib/findbugs/findbugsGUI.jar:/opt/local/src/hama/current/bin/../lib/findbugs/jsr305.jar:/opt/local/src/hama/current/bin/../lib/findbugs/plugin/coreplugin.jar:/opt/local/src/hadoop/current/conf:/opt/local/src/hadoop/current/conf
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/var/folders/pM/pMTPzO70Hc4Gd2FB24T-dU+++TM/-Tmp-/
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client environment:os.name=Mac
OS X
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:os.arch=x86_64
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:os.version=10.6.2
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client environment:user.name
=hadoop
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:user.home=/Users/hadoop
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Client
environment:user.dir=/opt/local/hadoop/jobs/hama
10/02/16 08:32:02 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2181 sessionTimeout=60000
watcher=org.apache.hadoop.hbase.client.hconnectionmanager$clientzkwatc...@76f8968f
10/02/16 08:32:02 INFO zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset
is false
10/02/16 08:32:02 INFO zookeeper.ClientCnxn: Attempting connection to server
localhost/0:0:0:0:0:0:0:1:2181
10/02/16 08:32:02 INFO zookeeper.ClientCnxn: Priming connection to
java.nio.channels.SocketChannel[connected local=/0:0:0:0:0:0:0:1%0:58679
remote=localhost/0:0:0:0:0:0:0:1:2181]
10/02/16 08:32:02 INFO zookeeper.ClientCnxn: Server connection successful
C = A + B
10/02/16 08:32:05 INFO matrix.AbstractMatrix: Initializing the matrix
storage.
10/02/16 08:32:07 INFO matrix.AbstractMatrix: Create Matrix
DenseMatrix_randwhqtl
10/02/16 08:32:07 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
10/02/16 08:32:07 WARN mapred.JobClient: No job jar file set. User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
10/02/16 08:32:08 INFO mapred.JobClient: Running job: job_201002160823_0003
10/02/16 08:32:09 INFO mapred.JobClient: map 0% reduce 0%
10/02/16 08:32:23 INFO mapred.JobClient: Task Id :
attempt_201002160823_0003_m_000000_0, Status : FAILED
java.lang.IndexOutOfBoundsException: v1.size != v2.size (25 != 23)
at
org.apache.hama.matrix.AbstractVector.checkComformantSize(AbstractVector.java:92)
at org.apache.hama.matrix.DenseVector.add(DenseVector.java:150)
at
org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:35)
at
org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:18)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/02/16 08:32:33 INFO mapred.JobClient: Task Id :
attempt_201002160823_0003_m_000000_1, Status : FAILED
java.lang.IndexOutOfBoundsException: v1.size != v2.size (25 != 23)
at
org.apache.hama.matrix.AbstractVector.checkComformantSize(AbstractVector.java:92)
at org.apache.hama.matrix.DenseVector.add(DenseVector.java:150)
at
org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:35)
at
org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:18)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/02/16 08:32:42 INFO mapred.JobClient: Task Id :
attempt_201002160823_0003_m_000000_2, Status : FAILED
java.lang.IndexOutOfBoundsException: v1.size != v2.size (25 != 23)
at
org.apache.hama.matrix.AbstractVector.checkComformantSize(AbstractVector.java:92)
at org.apache.hama.matrix.DenseVector.add(DenseVector.java:150)
at
org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:35)
at
org.apache.hama.matrix.algebra.MatrixAdditionMap.map(MatrixAdditionMap.java:18)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/02/16 08:32:54 INFO mapred.JobClient: Job complete: job_201002160823_0003
10/02/16 08:32:54 INFO mapred.JobClient: Counters: 3
10/02/16 08:32:54 INFO mapred.JobClient: Job Counters
10/02/16 08:32:54 INFO mapred.JobClient: Launched map tasks=4
10/02/16 08:32:54 INFO mapred.JobClient: Data-local map tasks=4
10/02/16 08:32:54 INFO mapred.JobClient: Failed map tasks=1
10/02/16 08:32:56 INFO client.HBaseAdmin: Disabled DenseMatrix_randwhqtl
10/02/16 08:32:56 INFO client.HBaseAdmin: Deleted DenseMatrix_randwhqtl
10/02/16 08:32:56 INFO zookeeper.ZooKeeper: Closing session:
0x126d6f5b8a90019
10/02/16 08:32:56 INFO zookeeper.ClientCnxn: Closing ClientCnxn for session:
0x126d6f5b8a90019
10/02/16 08:32:56 INFO zookeeper.ClientCnxn: Exception while closing send
thread for session 0x126d6f5b8a90019 : Read error rc = -1
java.nio.DirectByteBuffer[pos=0 lim=4 cap=4]
10/02/16 08:32:56 WARN zookeeper.ClientCnxn: Ignoring exception during
shutdown input
java.net.SocketException: Socket is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
10/02/16 08:32:56 INFO zookeeper.ClientCnxn: Disconnecting ClientCnxn for
session: 0x126d6f5b8a90019
10/02/16 08:32:56 INFO zookeeper.ZooKeeper: Session: 0x126d6f5b8a90019
closed
10/02/16 08:32:56 INFO zookeeper.ClientCnxn: EventThread shut down
run (rand, rand, add)
