Data retrieved is pretty small. Almost in bytes. It is basically user profile.
Might not be more than 256 bytes data per row
The configurations you are telling is mostly used in map reduce
The problem that we are facing is that the HBase thread pool is not working.
Below is the code for the same.
===========================================================================================================
import java.io.IOException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.util.Bytes;
public class TestPool{
private static final byte[] tableName ="sh_self_profiles".getBytes();
private static Configuration conf;
private static HTable table;
static{
conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", "10.64.0.156");
conf.set("hbase.zookeeper.property.clientPort","2181");
conf.set("hbase.master.port", "60000");
conf.setInt("hbase.htable.threads.max", 1000);
try {
table=new HTable(conf, tableName);
} catch (IOException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
//Here we are generating and hitting concurrent request
int nbThreads = 100;
ExecutorService execService =
Executors.newFixedThreadPool(nbThreads);
for (int i = 0; i < nbThreads; i++) {
execService.execute(new Runnable(){
public void run(){
try {
String
caller="9197100"+(int)(Math.random()*100000);
if(caller.length()==12){
TestPool.getOneRecord("sh_self_profiles",caller);
}
} catch (Throwable e) {
e.printStackTrace();
}
}//end-run
}//end-runnable
);//end-submit
}//end-for
}
//this method is used for getting result from hbase
public static void getOneRecord (String tableName, String rowKey)
throws IOException{
try{
long startTime=System.currentTimeMillis();
Get get = new Get(Bytes.toBytes(rowKey));
Result rs = table.get(get);
long endTime=System.currentTimeMillis();
System.out.println("total time "+(endTime-startTime));
System.out.println(Integer.parseInt(new
String(rs.getValue(Bytes.toBytes("sprofile"), Bytes.toBytes("male")))));
}catch (Exception e) {
e.printStackTrace();
}
}
}
========================================================================================================================
On 21-May-2014, at 8:56 am, Mikhail Antonov <[email protected]> wrote:
> How much data did you retrieve with single Get? if it's pretty small, you
> could try increasing this param.
>
> See 2.5.2.3 in http://hbase.apache.org/book/important_configurations.html
>
>
> 2014-05-20 20:21 GMT-07:00 Tahseen Jamal <[email protected]>:
>
>> Yes, have use the method you mentioned
>>
>> And the value of hbase.regionserver.handler.count is around 40
>>
>>
>>
>>
>> On 21-May-2014, at 8:38 am, Ted Yu <[email protected]> wrote:
>>
>>> Did you use the following method from HTable ?
>>>
>>> public Result[] get(List<Get> gets) throws IOException {
>>>
>>> What is the value for hbase.regionserver.handler.count ?
>>>
>>> Cheers
>>>
>>>
>>> On Tue, May 20, 2014 at 8:03 PM, Tahseen Jamal <[email protected]
>>> wrote:
>>>
>>>> Dear All,
>>>>
>>>>
>>>> I have 10 million records in a table. when i get single column value
>> from
>>>> hbase it takes around 10ms. My problem is when I hit 100 or more
>> concurrent
>>>> request the time slowly accumulates and increases to more than 400 ms
>>>> instead of completing in 10ms only. When 100 requests are hit linearly
>> each
>>>> one takes 10ms only. I am using hbase-0.96 version.
>>>>
>>>>
>>
>>
>
>
> --
> Thanks,
> Michael Antonov
