I think blocked dense matrix have a character of read-only. If we additionally put some elements, many blocks should be re-arranged.
On Mon, Nov 17, 2008 at 12:27 PM, Samuel Guo <[EMAIL PROTECTED]> wrote: > Why not we construct the matrix using block format? > It's a strange operation to block a DenseMatrix to a block matrix storage > format. > > BTW, blocking operation seems a expensive operation in Client. > > On Mon, Nov 17, 2008 at 10:48 AM, Edward J. Yoon <[EMAIL PROTECTED]>wrote: > >> When I tried to multiply the 1000 * 1000, I received below massage >> before start the mapreduce. >> >> This is a issue of blocking job. It'll be handled on HAMA-107. >> >> 08/11/17 11:08:09 INFO hama.AbstractMatrix: Initializing the matrix >> storage. >> 08/11/17 11:08:16 INFO hama.AbstractMatrix: Create Matrix >> DenseMatrix_randhzngs >> 08/11/17 11:08:16 INFO hama.AbstractMatrix: Create the 1000 * 1000 >> random matrix : DenseMatrix_randhzngs >> java.lang.RuntimeException: >> org.apache.hadoop.hbase.UnknownScannerException: >> org.apache.hadoop.hbase.UnknownScannerException: Name: >> 1659554672448075830 >> at >> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077) >> at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554) >> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888) >> >> at >> org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1195) >> at org.apache.hama.DenseMatrix.blocking(DenseMatrix.java:390) >> at >> org.apache.hama.examples.MatrixMultiplication.main(MatrixMultiplication.java:46) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) >> at >> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:141) >> at >> org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:30) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:155) >> at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >> at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68) >> Caused by: org.apache.hadoop.hbase.UnknownScannerException: >> org.apache.hadoop.hbase.UnknownScannerException: Name: >> 1659554672448075830 >> at >> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077) >> at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554) >> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888) >> >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) >> at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:513) >> at >> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:856) >> at >> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1155) >> at >> org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1192) >> ... 18 more >> >> >> On Fri, Nov 14, 2008 at 6:53 PM, Edward J. Yoon <[EMAIL PROTECTED]> >> wrote: >> > mat-mat mult operation time was dramatically reduced. (to 12 seconds >> > from 300 seconds on same condition) but code is not good (because by >> > me). please review this. >> > >> > And, we'll start the research about block decomposition. >> > -- >> > Best Regards, Edward J. Yoon @ NHN, corp. >> > [EMAIL PROTECTED] >> > http://blog.udanax.org >> > >> >> >> >> -- >> Best Regards, Edward J. Yoon @ NHN, corp. >> [EMAIL PROTECTED] >> http://blog.udanax.org >> > -- Best Regards, Edward J. Yoon @ NHN, corp. [EMAIL PROTECTED] http://blog.udanax.org
