Hi again,
overlooked your qeustiosn at the bottom. Sorry about that.
1) how many ways we can improve the performance(optimization) of Thrift
query to hbase.
Again, I suspect this part being more an HBase issue. But I may be wrong,
have no opportunity to test it.
2) how to cache the Thrift query response on Hbase. Can we force to cache
response on Hbase using thrift query(I couldn't find any additional
properties or attributes on Thrift scan/filter queries)
Thrift does not provide any chaching mechanisms, if that is what you are
looking for. However, you may want to serialize the data (using Thrift) and
put the serialized data into Memcached or Redis or the like. Start with
Redis, if you don't have any of these.
3) Is Thrift load balancing? how to manage load balancing of Thrift server
on Master node.
Same here. To some extent, load balancing this is out of scope. There are
some mechanisms, likeTThreadedServer or TThreadPool server, providing a very
simple, round robin load balancing across threads. If you want a load
balancer across processes or cluster nodes, although Thrift can help you to
achieve that, it does not provide anything like that out of the box.
JensG
-----Ursprüngliche Nachricht-----
From: Jens Geyer
Sent: Wednesday, January 15, 2014 9:00 PM
To: [email protected]
Subject: Re: Methods to improve the performance of Thrift (C#) query for
Hbase
And you are sure that it is not an HBase problem?
Just asking ...
-----Ursprüngliche Nachricht-----
From: Sanilpnair
Sent: Wednesday, January 15, 2014 3:31 PM
To: [email protected]
Subject: Methods to improve the performance of Thrift (C#) query for Hbase
Hi All,
I started Thrift coding (with c# and Hbase) recently. I need to fetch
realtime data from the Hbase using Thrift, my problem is that "performance
issue" with scanner - filter queries.
Please let me know the performance enhancement methods while using thrift
with c#. Sample code is given below:
---------------------------------------------------
string TableName="testtable";
Dictionary<byte[], byte[]> attributes = new Dictionary<byte[], byte[]>();
TScan scanFilter = new TScan();
scanFilter.FilterString =
ToByteArray("((RowFilter(<=,'binary:1053_1371222000')) AND
(RowFilter(>=,'binary:1053_1371217740'))) OR
(RowFilter(<=,'binary:1055_1371222000')) AND
(RowFilter(>=,'binary:1055_1371217740')))");
var scanner = _client.scannerOpenWithScan(ToByteArray(TableName),
scanFilter, attributes);
for (var entry = _client.scannerGet(scanner); entry.Count > 0; entry =
_client.scannerGet(scanner))
{
foreach (var rowResult in entry)
{
string rowkey =
Encoding.UTF8.GetString(rowResult.Row);
Console.Write("{0} => ", rowkey);
}
}
_client.scannerClose(scanner);
----------------------------------------------------
I tried scannerGetList also, single condition on filter is faster!
My question is that:
1) how many ways we can improve the performance(optimization) of Thrift
query to hbase.
2) how to cache the Thrift query response on Hbase. Can we force to cache
response on Hbase using thrift query(I couldn't find any additional
properties or attributes on Thrift scan/filter queries)
3) Is Thrift load balancing? how to manage load balancing of Thrift server
on Master node.
I am looking forward to hearing from you, hope you can help me.
Regards
Sanil
