bkietz commented on a change in pull request #41: ARROW-7580: [Website] 0.16 release post URL: https://github.com/apache/arrow-site/pull/41#discussion_r370340034
########## File path: _posts/2020-01-25-0.16.0-release.md ########## @@ -92,6 +92,33 @@ and result value, rather than taking a pointer-out function parameter ### C++: Datasets + +Added support for Arrow IPC files via `IPCFileFormat` (ARROW-7415), and the +`IN` and `IS_VALID` filter operators (ARROW-7185). + +Classes were renamed to avoid the repeated `Data` prefix. The core classes are +now `Dataset`, `Source`, `Fragment`, and `Partitioning` (ARROW-7498). The dataset +APIs now use `Result<T>` when returning both a Status and result value, rather +than taking a pointer-out function parameter (ARROW-7148). + +A discovery facility was added to create a `Dataset` and/or `Source` via the +`DatasetFactory` and `SourceFactory` interfaces. Notably `FileSystemSourceFactory` +can crawl directories to find the candidates files (`Fragment`) supporting by a +given `FileFormat`. The factories will also try to unify schemas, transparently +supporting missing/added columns to the final unified schema. (ARROW-6614, +ARROW-7061, ARROW-843, ARROW-7380) + +A partitioning facility was added to support partition pruning with predicate +pushdown (ARROW-6494). The partitioning extracts partition from the `Fragment`'s +path, e.g. `/data/year=2015/month=04/day=29` would extract the `year`, `month` +and `day` partitions. Partitions can injected in the schema and the RecordBatch Review comment: ```suggestion and `day` partitions. Partitions fields can injected into the schema and into the RecordBatch ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
