Dear Wiki user, You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.
The following page has been changed by udanax: http://wiki.apache.org/lucene-hadoop/Hbase/Matrix ------------------------------------------------------------------------------ [[TableOfContents(4)]] ---- - == HbaseLINA, a framework for large scale sparse linear algebra == + == HbaseLINA, a Framework for Large-scale Sparse Linear Algebra == Using [:Hbase:Hbase]'s Row,Column(Qualifier) two dimensional space, we are able to store large sparse matrix. [[BR]]The Auto-partitioned sparsity sub-structure will be efficiently managed, serviced by [:Hbase:Hbase]. - Also using iterative algorithms like conjugate gradient method on a parallel processing platform like MapReduce, - [[BR]]we should be able to implement '''High Performance''' and '''World's Largest''' Matrix Computations. + Row or Column operations can be done in linear time and algorithms such as structured Gaussian elimination + [[BR]]or iterative methods run in '''O(~-the number of non-zero elements in the matrix-~)''' time, [:Hbase:Hbase] Providing Both Vertical and Horizontal access. - The HbaseLINA application language will be added as sub-shell in [:Hbase/HbaseShell:Hbase Shell]. + Therefore, using iterative algorithms on a parallel processing platform like MapReduce, + [[BR]]we should be able to implement ''High Performance'' and ''World's Largest'' Matrix Computations. === Applications === @@ -31, +32 @@ == Storing and manipulating numeric, sparse matrices on Hbase == - A Sparse matrix is created by following the creation command with the keyword "Matrix". + === Scheduling Algorithm === - NOTE : + The scheduling algorithm is designed for driving the parallel execution of the factorization on on a MapReduce model. + === Hbase Shell Extension === - {{{ - difficulties in tuning sparse matrix. - - indirect and irregular, the inefficient data access + The HbaseLINA application language will be added as sub-shell in [:Hbase/HbaseShell:Hbase Shell]. - reordering for matrix partitioning (to create dense structure.) - - graph partitionning algorithms ( use hypergraph for non symmetric matrices ) - - TDF ordering - }}} - - - - === Hbase Shell Language Extension === - {{{ Hbase > lina; Hbase LINA version 0.0.1 Type 'help;' for help. - Hbase.LINA > M = Matrix('m_table','cf_1'); //Set up the matrix M from mapped matrix in hbase. + Hbase.LINA > M = Matrix('m_storage'); //Set up the matrix M from mapped matrix in hbase. Hbase.LINA > R = M.SVDdecomposition(); Hbase.LINA > U = R.getU(); - Hbase.LINA > Save U into Matrix('table_name','Othonormal_eigenvector_U'); + Hbase.LINA > Save U into Matrix('Othonormal_eigenvector_U'); ... Hbase.LINA > exit; @@ -68, +59 @@ == References == * [http://bebop.cs.berkeley.edu/oski/ OSKI], optimized sparse kernel interface (OSKI) library + * [http://icl.cs.utk.edu/iclprojects/pages/files/sans/yelick-bebop.pdf Automatic Performance Tuning of Sparse Matrix Kernels], August 7, 2002 * Parallel Conjugate Gradients Assignment, a parallel implementation of the conjugate gradient algorithm * ScaLAPACK, a library of high-performance linear algebra routines for distributed-memory message-passing MIMD computers * Scheduling algorithms for parallel Gaussian elimination withcommunication costs, Amoura, A.K.; Bampis, E.; Konig, J.-C.
