keksmd opened a new pull request, #961:
URL: https://github.com/apache/incubator-graphar/pull/961
### Reason for this PR
Part of #947, tracked by #959. The reader and writer verticals both need one
format-neutral
description of a row group before either can name a file format. Without it,
the Parquet backend and the GraphAr reader would each invent their own row
representation, and the two would have to be reconciled later.
### What changes are included in this PR?
A new `graphar-io-api` module holding the neutral batch description plus the
write half of the physical IO boundary:
- `ColumnType` — recursive, format-neutral type (scalars plus `LIST`).
- `Field`, `Schema` — a named nullable column and an ordered, duplicate-free,
immutable column list.
- `Row`, `RecordBatch`, `BatchCursor` — a row whose values follow one
documented
Java mapping, a finite batch sharing a schema, and a closeable cursor.
- `WriteRequest`, `WriteMode`, `PhysicalWriter` — a URI, a schema, and a
target
disposition, and the writer that drains a cursor into that target.
The module has no dependency on any other GraphAr module, so a format backend
can implement it without inheriting the metadata or storage layers.
The read half of the boundary (`ReadRequest`, projection, filter, row range,
capability reporting, `PhysicalReader`) is deliberately left out and will
follow
as its own PR against this base, so that the two halves can be reviewed
independently.
### Are these changes tested?
Yes, 27 tests. The value types are pinned directly; the batch, cursor, and
writer contracts are pinned against an in-memory reference implementation
that
stands in for a real format backend, covering the documented value mapping
for
every column kind, cursor exhaustion, cursor ownership, schema mismatch, and
both write dispositions.
```
mvn --no-transfer-progress -pl io-api clean verify
Tests run: 27, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS
```
### Are there any user-facing changes?
A new module and package `org.apache.graphar.io`. Nothing existing changes.
## Checklist
- [x] I have performed a self-review of my own code.
- [ ] I have formatted my own code using `make cpplint` before submitting
when changed files are in the `cpp` directory.
- [x] I have performed `pre-commit run` before commit the changed files.
- [x] I have added tests to prove my changes are effective.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
