keksmd opened a new pull request, #983: URL: https://github.com/apache/incubator-graphar/pull/983
### Reason for this PR Part of #978: the first read vertical of #947. It turns a GraphAr dataset's `ordered_by_source` adjacency and edge properties into edge and neighbor streams without JNI. Vertex property scans are a stated follow-up and are not in this PR. ### What changes are included in this PR? - New `graphar-reader` module. `GraphReader` sits over `graphar-info`, `graphar-core` and the `PhysicalReader` contract from `graphar-io-api`. `graphar-io-parquet` is a test dependency only. - `OrderedSourceEdgeReader.neighbors(vertexId)`: `OrderedAdjacencyResolver` locates the offset chunk, only the two bounding offsets are read, and only that row range of the adjacency chunks it touches is requested. - `scanEdges()` streams every topology row chunk by chunk. - `scanEdges(properties)` and per-vertex edge property cursors read the property chunks with the same row ranges and join them by row position. A chunk that returns a different row count than its range fails the read. - `java-sdk.yml` runs the new module. ### Are these changes tested? Yes, against the canonical `testing/ldbc_sample` Parquet fixtures, compared with values read through Arrow C++. Per-vertex reads are checked by recording physical requests and counting storage opens. The Arrow C++ edge chunks have no Parquet Offset Index, which `graphar-io-parquet` currently needs for a partial row-group range, so per-vertex tests read a copy rewritten by `ParquetPhysicalWriter`. That io-parquet gap is tracked in #981. `mvn -pl reader -am clean verify`: reader 8 tests, 0 failures. ### Are there any user-facing changes? New module `graphar-reader`. No changes to existing APIs. ## Checklist - [x] I have performed a self-review of my own code. - [ ] I have formatted my own code using `make cpplint` before submitting when changed files are in the `cpp` directory. - [x] I have performed `pre-commit run` before commit the changed files. - [x] I have added tests to prove my changes are effective. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
