Hi Sungwook, Can you create a different directory with a few files in each sub directory, but use 2012, 2013, 2014instead of year=2012, etc. Might be a good test to see if the directory naming structure of year=xxxx is tripping up Drill on directory pruning.
—Andries > On Aug 23, 2015, at 9:47 AM, Kristine Hahn <[email protected]> wrote: > > If you set up your data in directories like the log data in the > Querying Directories example on > http://drill.apache.org/docs/querying-directories, which uses WHERE > dir0='2013' LIMIT 10 in the query, and you are having intermittent > Table Not Found results, look for hidden files in the directory you > are querying. The files must be compatible--they must have comparable > data types and columns in the same order. Hidden files that do not > have comparable data types can cause a Table Not Found error. > Kristine Hahn > Sr. Technical Writer > 415-497-8107 @krishahn skype:krishahn > > > > On Sun, Aug 23, 2015 at 9:01 AM, USC <[email protected]> wrote: >> Hi Sungwook, >> In your where clause, you only need to say year=2012. >> >> The directory column (e.g., dir0) is used when users query a directory. >> >> >>> On Aug 23, 2015, at 7:07 AM, Sungwook Yoon <[email protected]> wrote: >>> >>> Hi, >>> >>> I am trying to use Hive parquet stored files partitioned by some column. >>> So, the directory structure is partitioned with the column. >>> >>> The column is actually year. >>> Let's say there are 5 years, so dir0 are like year=2010, >>> year=2011,year=2012,year=2013,year=2014 >>> >>> We did like following >>> select * from dfs.root.`/user/hive/warehouse/table` d where d.dir0 = >>> 'year=2012'; >>> >>> I get nothing. >>> Apparently, there are parquet files in the directory though. >>> >>> Sometimes it picks up e.g., year=2010, but not year=2012.. >>> >>> Where am I going wrong with this? >>> >>> Thanks, >>> >>> Sungwook
