Re: Filters on Arrow record batch

[Wes McKinney]

hi Animesh -- it does not yet, but the idea has come up on occasion.
You are welcome to propose additions to the format for including
statistics in a stream of record batch messages (these could possibly
be embedded in the main RecordBatch metadata or sent as a separate
message).

As an aside, I just opened
https://issues.apache.org/jira/browse/ARROW-2022 to think about the
idea of sending along arbitrary extra metadata with a record batch
message.

- Wes

On Sat, Jan 20, 2018 at 6:07 AM, Animesh Trivedi
 wrote:
> Hi all,
>
> Is it possible to have push-down filters on Arrow record batches while
> reading data in? Something like what parquet have.
>
> Does Arrow maintain any per batch statistics?
>
> Thanks
> --
> Animesh

Filters on Arrow record batch

[Animesh Trivedi]

Hi all,

Is it possible to have push-down filters on Arrow record batches while
reading data in? Something like what parquet have.

Does Arrow maintain any per batch statistics?

Thanks
--
Animesh