hi Animesh -- it does not yet, but the idea has come up on occasion. You are welcome to propose additions to the format for including statistics in a stream of record batch messages (these could possibly be embedded in the main RecordBatch metadata or sent as a separate message).
As an aside, I just opened https://issues.apache.org/jira/browse/ARROW-2022 to think about the idea of sending along arbitrary extra metadata with a record batch message. - Wes On Sat, Jan 20, 2018 at 6:07 AM, Animesh Trivedi <animesh.triv...@gmail.com> wrote: > Hi all, > > Is it possible to have push-down filters on Arrow record batches while > reading data in? Something like what parquet have. > > Does Arrow maintain any per batch statistics? > > Thanks > -- > Animesh
