bq. Scan.setRowCaching() I think you meant Scan.setCaching()
On Mon, Mar 17, 2014 at 1:28 PM, Vladimir Rodionov <vrodio...@carrieriq.com>wrote: > > HBase RegionServer does scanning in batches, client requests next batch > from server > and server reads and merge the data from cache/disk. You can control batch > data size by setting both: > > Scan.setRowCaching(number of rows to send in one RPC request) > > Technically speaking, this allows you to control LIMIT from the client > side. Your overhead will never be larger than the limit set by > setRowCaching. > > Best regards, > Vladimir Rodionov > Principal Platform Engineer > Carrier IQ, www.carrieriq.com > e-mail: vrodio...@carrieriq.com > > ________________________________________ > From: Weiping Qu [q...@informatik.uni-kl.de] > Sent: Monday, March 17, 2014 12:19 PM > To: dev@hbase.apache.org > Subject: Re: 答复: setMaxResultSize method in Scan > > I am doing a mult-thread(100) scan test over hbase. > If one request with given key-range matches a large number of > correspoding rows in hbase, my request is waiting for this scan to > complete. > The throughput is really slow. > For test purpose, I'd like to use LIMIT to reduce the time on scanning > and transferring results back from hbase to increase the throughput. > Do you think the "hbase.client.scan.max.result.size" or > setMaxResultSize(in bytes) could help HBase to stop scan at the LIMIT > before scanning complete corresponding rows? > > As you mentioned that there is no query optimizer in HBase, I assume > that region servers will not stop scanning the rows in this key-range in > this case until it gets all the results and limit the results to max > size which is sent to the client. > If so, there is not much I can do to compare the throughput with that in > relational databases like MySQL. > > Thanks, > Cheers. > > Limit clause in SQL Select statement makes sense because it allows query > optimizer to plan accordingly. > > It does not make sense in HBase as since there is no query planner and > or optimization involved during > > scanning HBase table. You can easily mimic this functionality on a > client side (I mean - limit). > > > > Best regards, > > Vladimir Rodionov > > Principal Platform Engineer > > Carrier IQ, www.carrieriq.com > > e-mail: vrodio...@carrieriq.com > > > > ________________________________________ > > From: Weiping Qu [q...@informatik.uni-kl.de] > > Sent: Monday, March 17, 2014 4:48 AM > > To: dev@hbase.apache.org > > Subject: Re: 答复: setMaxResultSize method in Scan > > > > Thanks. > > > > I'd like to assume that setMaxResultSize is equivalent to the SQL Limit > > clause as expected which is specified each time a SQL statement is > > executed . > > Now through "hbase.client.scanner.max.result.size" can the limitation of > > number of row returned only apply to all the scanner instances. > > I am wondering why the setMaxResultSize is removed now. > > > >> No such method for Scan in 0.94.x. > >> > >> If you want to set the max result size for a scan, you can achieve this > by setting the "hbase.client.scanner.max.result.size" configuration, the > default for which is Long.MAX_VALUE (no limited) > >> ________________________________________ > >> 发件人: Weiping Qu [q...@informatik.uni-kl.de] > >> 发送时间: 2014年3月17日 18:50 > >> 收件人: dev@hbase.apache.org > >> 主题: setMaxResultSize method in Scan > >> > >> Hello, > >> > >> I could not find the method setMaxResultSize(long m) > >> ( > http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html) > >> in my Scanclass (0.94.13 version). > >> Can anyone help me? Thanks > >> > >> Weiping > > > > -- > > Mit freundlichen Grü?en / Kind Regards > > > > *Weiping Qu* > > > > University of Kaiserslautern > > Department of Computer Science > > Heterogeneous Information Systems Group > > P.O. Box 3049 > > 67653 Kaiserslautern, Germany > > > > Email: qu (at) informatik.uni-kl.de > > Phone: +49 631 205 3264 > > Fax: +49 631 205 3299 > > Room: 36/331 > > > > Confidentiality Notice: The information contained in this message, > including any attachments hereto, may be confidential and is intended to be > read only by the individual or entity to whom this message is addressed. If > the reader of this message is not the intended recipient or an agent or > designee of the intended recipient, please note that any review, use, > disclosure or distribution of this message or its attachments, in any form, > is strictly prohibited. If you have received this message in error, please > immediately notify the sender and/or notificati...@carrieriq.com and > delete or destroy any copy of this message and its attachments. > > > -- > Mit freundlichen Grü?en / Kind Regards > > *Weiping Qu* > > University of Kaiserslautern > Department of Computer Science > Heterogeneous Information Systems Group > P.O. Box 3049 > 67653 Kaiserslautern, Germany > > Email: qu (at) informatik.uni-kl.de > Phone: +49 631 205 3264 > Fax: +49 631 205 3299 > Room: 36/331 >
