Hi,
At lease the table size must be greater than 5GB to use the index for
filter pushdown. Otherwise you have to comment the checkQuerySize method.
Cheers,
Mahsa
On Mon, Jul 30, 2012 at 11:12 AM, Ablimit Aji <abli...@gmail.com> wrote:
> I have written a custom index handler and wanted to test it. However hive
> is not using it.
> So I test with simple table (pokes (int foo, string bar)) which comes with
> hive distribution for testing purpose.
> Then I created a compact index and set the set
> hive.optimize.index.filter=true;
> However, upon checking the log info, it seems hive is still not using the
> index.
> So, what is the problem ?
> The query I issued is as follow: select foo from pokes WHERE foo=498 ;
>
> Below is the log info I got after issuing the query.
>
> 12/07/26 12:25:17 INFO index.IndexWhereProcessor: Processing predicate for
> index optimization
> 12/07/26 12:25:17 INFO index.IndexWhereProcessor: (foo = 498)
> 12/07/26 12:25:17 INFO metastore.HiveMetaStore: 0: get_table : db=default
> tbl=pokes_idx
> 12/07/26 12:25:17 INFO hive.log: DDL: struct pokes_idx { i32 foo, string
> _bucketname, list _offsets}
> 12/07/26 12:25:17 INFO index.IndexWhereProcessor: checking index
> staleness...
> 12/07/26 12:25:17 INFO index.IndexWhereProcessor: 1342465077455
> 12/07/26 12:25:17 INFO index.IndexWhereProcessor: 1342465077455
> 12/07/26 12:25:17 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 12/07/26 12:25:17 WARN snappy.LoadSnappy: Snappy native library not loaded
>
