Would be really nice to have that. I had plans to do it at some points but didn't get a chance to pursue them. Not only it is useful for Parquet, we can also use it with other formats. we will need a new rewrite rule that pushes projection to datasources. This should be generic enough to be used with internal datasets as well (assuming we support PAX at some point).
I am sure there are many hidden obstacle to writing this rewrite rule but I think that it can be done in a week. Cheers, Abdullah. On Sat, Aug 27, 2016 at 11:37 PM, Mike Carey <[email protected]> wrote: > It would be INCREDIBLY helpful be able to do this (also for internal > sources) - e.g., even our basic index access operators could optionally > accept a filter to apply to the records they'd otherwise simply return. > This could especially useful when combined with limit pushdown, I would > think, too. > > > On 8/26/16 3:44 PM, Wail Alkowaileet wrote: > >> Hi AsterixDBers. >> >> Is there any easy way to push-down filter to an external source (in my >> case >> Parquet) without being too intrusive ? >> >> This can perform way faster than STREAM-SELECT, as parquet can potentially >> skip Row Groups while scanning. >> > >
