Drill applies LIMIT filtering at row group level. For LIMIT n, it
will scan the first m row groups that have at least n rows, and
discard the rest of row groups. In your case, since you have only 1
row group, it does not have any row group filtering for LIMIT 1.
I'm not sure how 32767 comes from. It's possibly that's the size of
the first batch, depending on the column data in your file.
On Mon, Sep 19, 2016 at 7:09 AM, Veera Naranammalpuram
> Does anyone know how and if the LIMIT push down to Parquet file works?
> I have a parquet file with 53K records in 1 row group. When I run a SELECT
> * from <table> LIMIT 1, I see the Parquet reader operator process 32768
> records. I would have expected either 1 or 53K. So questions;
> 1) Does the Parquet MR library offer the ability to push down LIMITs to
> Parquet files? From the above, the answer looks like yes.
> 2) If so, how does Drill come up with the magic number 32767? Is there a
> way I can make it read just 1 row if the query is a LIMIT 1?
> Veera Naranammalpuram
> Product Specialist - SQL on Hadoop
> *MapR Technologies (www.mapr.com <http://www.mapr.com>)*
> *(Email) vnaranammalpu...@maprtech.com <naranammalpu...@maprtech.com>*
> *(Mobile) 917 683 8116 - can text *
> *Timezone: ET (UTC -5:00 / -4:00)*