ARROW-5916 ARROW-6836/6837 These are of particular interest to me because they enable recordbatch "incrementalism" which is useful for streaming applications:
ARROW-5916 allows a recordbatch to pre-allocate space for future records that have not yet been populated, making it safe for readers to consume the partial batch. ARROW-6836/6837 allows a file of record batches to be extended at the end, without re-writing the beginning, while including the idea that the custom_metadata may change with each update. (custom_metadata in the Schema is not a good candidate because Schema also appears at the beginning of the file.) While these are not blockers for me quite yet, they soon will be! If I wanted to ensure that these are in 1.0, what is my deadline for implementation and test cases? Can such a note be made on the wiki? Should I change the priority in Jira? Thanks, John On Wed, Oct 9, 2019 at 2:57 PM Neal Richardson <neal.p.richard...@gmail.com> wrote: > Congratulations everyone on 0.15! I know a lot of hard work went into > it, not only in the software itself but also in the build and release > process. > > Once you've caught your breath from the release, we should start > thinking about what's in scope for our next release, the big 1.0. To > get us started (or restarted, since we did discuss 1.0 before the > flatbuffer alignment issue came up), I've created > https://cwiki.apache.org/confluence/display/ARROW/Arrow+1.0.0+Release > based on our past release wiki pages. > > A good place to begin would be to list, either in "blocker" Jiras or > bullet points on the document, the key features and tasks we must > resolve before 1.0. For example, I get the sense that we need to > overhaul the documentation, but that should be expressed in a more > concrete, actionable way. > > Neal >
