ARROW-6837 (which, er, includes ARROW-6836) and ARROW-5916 have PRs.
Would appreciate some feedback. I will finish the Python part of 6837 when
I know I'm on the right track.
Thanks,
John
On Thu, Oct 10, 2019 at 9:54 AM John Muehlhausen wrote:
> The format change is ARROW-6836 ... add a custo
The format change is ARROW-6836 ... add a custom_metadata:[KeyValue] field
to the Footer table in File.fbs
The other change (slicing a recordbatch to honor RecordBatch.length rather
than array length if the former is smaller) will hopefully not affect the
format.
On Wed, Oct 9, 2019 at 11:55 PM
Hi John,
Since the 1.0.0 release is focused on Format stability, probably the
only real "blockers" will be ensuring that we have hardened multiple
implementations (in particular C++ and Java) of the columnar format as
specified with integration tests to prove it. The issues you listed
sound more l
ARROW-5916
ARROW-6836/6837
These are of particular interest to me because they enable recordbatch
"incrementalism" which is useful for streaming applications:
ARROW-5916 allows a recordbatch to pre-allocate space for future records
that have not yet been populated, making it safe for readers to c
Congratulations everyone on 0.15! I know a lot of hard work went into
it, not only in the software itself but also in the build and release
process.
Once you've caught your breath from the release, we should start
thinking about what's in scope for our next release, the big 1.0. To
get us started
