Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.
The "SpMV" page has been changed by Mikalai Parafeniuk: http://wiki.apache.org/hama/SpMV?action=diff&rev1=11&rev2=12 /user/hduser/spmv/vector-seq /user/hduser/spmv/vector-txt }}} - Suffix `seq` denotes that directory contains sequence files. Suffix `txt` denotes that directory contains human-readable text files. + Suffix `seq` denotes that directory contains sequence files. Suffix `txt` denotes that directory contains human-readable text files. <<BR>> + Also I defined some shell variables in .bashrc file of hadoop user to simplify following scripts. + {{{ + export HAMA_EXAMPLES=$HAMA_HOME/hama-examples*.jar + export SPMV=/user/hduser/spmv + }}} + First variable allows fast access to jar with hama examples, which plased in hama home directory, second variable is prefix in HDFS for tests in this tutorial. If you not defined this variables just substitute appropriate values into following scripts. ==== Usage with RandomMatrixGenerator ==== `RandomMatrixGenerator` as a `SpMV` works with sequence file format. So, to multiply random matrix with random vector we will do the following: generate matrix and vector; convert matrix, vector and result to text file; view matrix, vector and result. This sequence is described by the following code snippet: {{{ - 1: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar rmgenerator /user/hduser/spmv/matrix-seq 4 4 0.3 4 - 2: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar rmgenerator /user/hduser/spmv/vector-seq 1 4 0.8 1 - 3: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar spmv /user/hduser/spmv/matrix-seq /user/hduser/spmv/vector-seq /user/hduser/spmv/result-seq 4 - 4: ../hadoop/bin/hadoop dfs -rmr /user/hduser/spmv/result-seq/part - 5: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar matrixtotext /user/hduser/spmv/matrix-seq /user/hduser/spmv/matrix-txt - 6: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar matrixtotext /user/hduser/spmv/vector-seq /user/hduser/spmv/vector-txt - 7: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar matrixtotext /user/hduser/spmv/result-seq /user/hduser/spmv/result-txt + 0: hadoop dfs -rmr $SPMV/*/* + 1: hama jar $HAMA_EXAMPLES rmgenerator $SPMV/matrix-seq 6 6 0.4 4 + 2: hama jar $HAMA_EXAMPLES rmgenerator $SPMV/vector-seq 1 6 0.9 4 + 3: hama jar $HAMA_EXAMPLES spmv $SPMV/matrix-seq $SPMV/vector-seq $SPMV/result-seq 4 + 4: hadoop dfs -rmr $SPMV/result-seq/part + 5: hama jar $HAMA_EXAMPLES matrixtotext $SPMV/matrix-seq $SPMV/matrix-txt + 6: hama jar $HAMA_EXAMPLES matrixtotext $SPMV/vector-seq $SPMV/vector-txt + 7: hama jar $HAMA_EXAMPLES matrixtotext $SPMV/result-seq $SPMV/result-txt - 8: ../hadoop/bin/hadoop dfs -cat /user/hduser/spmv/matrix-txt/* + 8: hadoop dfs -cat /user/hduser/spmv/matrix-txt/* - 0 4 1 2 0.09775514911904559 - 3 4 1 1 0.22718006778335464 - 1 4 1 1 0.796916052801057 - 3 4 1 2 0.9680719390036476 - 2 4 1 0 0.44269679525022 + 0 6 3 5 0.24316243288531214 2 0.638622414091597 3 0.5480468710898891 + 3 6 2 5 0.5054043538570098 2 0.03911646523753309 + 1 6 3 4 0.5077528966368161 5 0.5780340816354201 3 0.4626752204959449 + 4 6 2 1 0.6512355661856207 4 0.08804976645891671 + 2 6 2 4 0.7200271909735554 1 0.3510851368183805 + 5 6 2 2 0.5848717104309032 3 0.0889791409798859 + - 9: ../hadoop/bin/hadoop dfs -cat /user/hduser/spmv/vector-txt/* + 9: hadoop dfs -cat /user/hduser/spmv/vector-txt/* - 0 4 4 0 0.3111172930766064 1 0.27242674637140374 2 0.11394746764097863 3 0.0 + 0 6 6 0 0.3365077672167889 1 0.17498609722570935 2 0.32806410950648845 3 0.6016567879100464 4 0.786158850847722 5 0.6856872945972037 - 10: ../hadoop/bin/hadoop dfs -cat /user/hduser/spmv/result-txt/* + 10: hadoop dfs -cat /user/hduser/spmv/result-txt/* - 0 5 5 0 0.011138951690981488 1 0.21710124739573375 2 0.1377306285919371 3 0.11030934594375758 4 0.0 + 0 6 6 0 0.7059786044267415 1 1.0738967463653346 2 0.6274907669206862 3 0.35938205240905363 4 0.18317827331814918 5 0.24541032101100438 }}} - Line 1-2: Generation of input matrix and vector.<<BR>> + Line 0: Clean up of directories related to SpMV tests.<<BR>> + Line 1-2: Generation of input matrix and vector. In this example we test 6x6 matrix and 1x6 vector multiplication<<BR>> Line 3: SpMV algorithm.<<BR>> Line 4: Deletion of part files from output directory at line 4. NOTE: `matrixtotext` will fail if this step will not be performed, because `result-seq` will containg part folder and `matrixtotext` don't know how to deal with it yet.<<BR>> Line 5-7: Convertion of input matrix, input vector and result to text format.<<BR>> - Line 8-10: Showing the result. NOTE: currently SpMV is buggy - you can see, that output vector has length of 5, and the last cell of vector has incorrect value. This will be fixed soon. + Line 8-10: Showing the result. ==== Usage with arbitrary text files ==== SpMV works with `SequenceFile`, so we need to provide tools to convert input and output of SpMV between sequence file format and text format. These tools are `matrixtoseq` and `matrixtotext`. This programs are included in example driver, so they can be launched like any other example. `matrixtoseq` converts matrix, represented in text file to sequence file format. Also this program gives choice to choose target writable: `DenseVectorWritable` and `SparseVectorWritable`.
