Hi, Just want to make sure I understood Eric's question. Eric, were you considering a run-time call to get this information?
If so, as Qifan points out, this information is available to the compiler. We just would have to get it to the run-time somehow. Usual route is through the appropriate TDB generated by the compiler and read by the executor. Dave -----Original Message----- From: Qifan Chen [mailto:[email protected]] Sent: Wednesday, August 5, 2015 4:04 PM To: dev <[email protected]> Subject: Re: avoiding a JNI call to getHbaseTableInfo? Hi Eric, You can set the CQD Hbase_index_level to a positive integer value (such as 2) to completely bypass the JNI call. Note that the JNI call also reads in the index level, which is a function of the Hfiles in use. Since we cache NATable, which is an in-memory representation of a HBase table, the JNI cost is only incurred once for all references to a table, per SQL session. If the table is redefined, we will re-read the definition and call the JNI again. Thanks --Qifan On Wed, Aug 5, 2015 at 5:47 PM, Eric Owhadi <[email protected]> wrote: > For the small scanner feature, I first hardcoded the default > HBaseBlock size (64KB) in the check to see if small scanner is good > candidate to enable or not. > > Then struggling with the regression tests and the modifications needed > to pass, I realized that it is better to have a feature totally > completed to avoid the cost of dealing with regression every time you > want to improve it. > > So I am trying to remove the hardcode for HBase block size, and > realized there is a function getHbaseTableInfo that will do the job of > getting HBase block size and index level. BUT, this function is doing > a JNI call to the HBase layer. Look like expensive to call for every > scan query… will likely lose the gain of small scanner if I put this call > in the middle. > > I was wondering if this Hbase block size per table is not already > cached in memory so that I can access it without fear of performance > cost? I guess it is table meta data, and I think this is supposed to > already be cached if I recall a previous email thread? > > Thanks in advance for the help, > > Eric > -- Regards, --Qifan
