That is what I am trying to get: the blocksize from the TDB... still looking where that is :-)... Now that I know blocksize is there, forget about the other question, browsing the code I only saw the runtime call, and was afraid the data was not in TDB, but look like you are saying it is, so this is perfect, I just have to find it...
On Wed, Aug 5, 2015 at 6:28 PM, Dave Birdsall <[email protected]> wrote: > Hi, > > Just want to make sure I understood Eric's question. Eric, were you > considering a run-time call to get this information? > > If so, as Qifan points out, this information is available to the compiler. > We just would have to get it to the run-time somehow. Usual route is > through > the appropriate TDB generated by the compiler and read by the executor. > > Dave > > -----Original Message----- > From: Qifan Chen [mailto:[email protected]] > Sent: Wednesday, August 5, 2015 4:04 PM > To: dev <[email protected]> > Subject: Re: avoiding a JNI call to getHbaseTableInfo? > > Hi Eric, > > You can set the CQD Hbase_index_level to a positive integer value (such as > 2) to completely bypass the JNI call. Note that the JNI call also reads in > the index level, which is a function of the Hfiles in use. > > Since we cache NATable, which is an in-memory representation of a HBase > table, the JNI cost is only incurred once for all references to a table, > per > SQL session. If the table is redefined, we will re-read the definition and > call the JNI again. > > Thanks --Qifan > > On Wed, Aug 5, 2015 at 5:47 PM, Eric Owhadi <[email protected]> wrote: > > > For the small scanner feature, I first hardcoded the default > > HBaseBlock size (64KB) in the check to see if small scanner is good > > candidate to enable or not. > > > > Then struggling with the regression tests and the modifications needed > > to pass, I realized that it is better to have a feature totally > > completed to avoid the cost of dealing with regression every time you > > want to improve it. > > > > So I am trying to remove the hardcode for HBase block size, and > > realized there is a function getHbaseTableInfo that will do the job of > > getting HBase block size and index level. BUT, this function is doing > > a JNI call to the HBase layer. Look like expensive to call for every > > scan query… will likely lose the gain of small scanner if I put this call > > in the middle. > > > > I was wondering if this Hbase block size per table is not already > > cached in memory so that I can access it without fear of performance > > cost? I guess it is table meta data, and I think this is supposed to > > already be cached if I recall a previous email thread? > > > > Thanks in advance for the help, > > > > Eric > > > > > > -- > Regards, --Qifan >
