RE: issue about search speed and rowkey

[Lu, Wei]

decrease cache size (say, 1000)  and increase batch or just set it as default 
if #qualifiers in a row is not too many.


-----Original Message-----
From: ch huang [mailto:justlo...@gmail.com] 
Sent: Wednesday, August 07, 2013 5:18 PM
To: user@hbase.apache.org
Subject: issue about search speed and rowkey

hi,all:

        i have problem in running following code,MoveData method is used to
get data from source table ,and modify each row rowkey ,and insert into
destination table,and i always get error,anyone can help?

 public static void writeRow(HTablePool htp,String tablename, String
rowkey,String cf,String col,String value) {
        try {
         HTableInterface table = htp.getTable(Bytes.toBytes(tablename));
            Put put = new Put(Bytes.toBytes(rowkey));
                put.add(Bytes.toBytes(cf),
                  Bytes.toBytes(col),
                  Long.parseLong(rowkey),
                  Bytes.toBytes(value));

                table.put(put);
            table.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

public static void MoveData(String src_t,String dest_t){

    try{
     HTable tabsrc = new HTable(conf, src_t);
     HTable tabdest = new HTable(conf, dest_t);
     tabsrc.setAutoFlush(false);
     tabdest.setAutoFlush(false);
     HTablePool tablePool = new HTablePool(conf, 5);
     Scan scan = new Scan();
     scan.setCaching(10000);
        scan.setBatch(10);
        ResultScanner rs = tabsrc.getScanner(scan);

           for (Result r : rs){
            ArrayList al = new ArrayList();
            HashMap hm = new HashMap();
            for (KeyValue kv : r.raw()){

             hm.put(new String(kv.getQualifier()), new
String(kv.getValue()));
             al.add(new String(kv.getQualifier()));
            }

               for (int i = 0; i < al.size(); i++) {

writeRow(tablePool,dest_t,hm.get("date").toString(),"info",al.get(i).toString(),hm.get(al.get(i)).toString());
               }
         }

           rs.close();
           tabsrc.close();
           tabdest.close();

    }catch(IOException e){
     e.printStackTrace();
    }

   }



2013-08-07 16:43:31,250 WARN  [main] conf.Configuration
(Configuration.java:warnOnceIfDeprecated(824)) - hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
java.lang.RuntimeException:
org.apache.hadoop.hbase.client.ScannerTimeoutException: 123891ms passed
since the last invocation, timeout is currently set to 120000
 at
org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:44)
 at com.testme.demo.HBaseTest.MoveData(HBaseTest.java:186)
 at com.testme.demo.HBaseTest.main(HBaseTest.java:314)
Caused by: org.apache.hadoop.hbase.client.ScannerTimeoutException: 123891ms
passed since the last invocation, timeout is currently set to 120000
 at
org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:283)
 at
org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:41)
 ... 2 more
Caused by: org.apache.hadoop.hbase.UnknownScannerException:
org.apache.hadoop.hbase.UnknownScannerException: Name: -1792412350530007203
 at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2464)
 at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
 at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)

 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
Source)
 at java.lang.reflect.Constructor.newInstance(Unknown Source)
 at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
 at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:143)
 at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:42)
 at
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:163)
 at
org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:274)
 ... 3 more