This is an automated email from the ASF dual-hosted git repository.
SYaoJun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-graphar.git
The following commit(s) were added to refs/heads/main by this push:
new b3e73271 feat(java): add chunk and edge-range primitives (#968)
b3e73271 is described below
commit b3e732714256b9f1e29501fe19a354a59975a908
Author: alex <[email protected]>
AuthorDate: Wed Sep 9 13:51:46 2026 +0300
feat(java): add chunk and edge-range primitives (#968)
* feat(java): add chunk and edge-range primitives
Provide the storage-free arithmetic layer needed by future ordered
adjacency readers without coupling it to GraphAr metadata or physical IO.
Relates to #967.
Rejected: resolving metadata layouts or reading offsets in this value-only
slice.
* feat(java): resolve ordered adjacency ranges
Complete the core-only ordered-adjacency contract: validate offset chunks,
select exact half-open edge chunk ranges, and delegate URI resolution to
immutable GraphAr metadata.
Relates to #967.
Rejected: reading Parquet offsets in graphar-core; io-parquet remains
behind the open #961 API change.
* ci(java): build the core module the workflow already watches
The workflow's path filters already list maven-projects/core/**, so a change
under core triggers the job, but SDK_MODULES left the module out and the
reactor never compiled or tested it. A green run therefore proved nothing
about core.
Add core to the module list so the same job that guards storage and io-api
guards it too. The fixture-backed test skips itself when the canonical
testing dataset is absent, so the job stays green without the submodule.
* Make an offset chunk say which vertex chunk it came from
An ordered-layout resolve took any OffsetChunk and indexed it with an
offset computed from a different vertex chunk, silently returning a
plausible but wrong edge range. The chunk now carries the vertex chunk
it was read from, and resolve rejects a mismatch before using it.
Chunk arithmetic in EdgeRange.edgeChunks now delegates to ChunkMath
instead of repeating the division, and ChunkRange, EdgeRange and
OffsetLocation gained value equality plus readable toString so tests can
assert whole values rather than field by field.
---
.github/workflows/java-sdk.yml | 2 +-
maven-projects/core/pom.xml | 88 ++++++++++
.../java/org/apache/graphar/core/ChunkMath.java | 79 +++++++++
.../java/org/apache/graphar/core/ChunkRange.java | 105 ++++++++++++
.../java/org/apache/graphar/core/EdgeRange.java | 123 ++++++++++++++
.../java/org/apache/graphar/core/OffsetChunk.java | 114 +++++++++++++
.../org/apache/graphar/core/OffsetLocation.java | 89 ++++++++++
.../graphar/core/OrderedAdjacencyResolver.java | 92 +++++++++++
.../org/apache/graphar/core/ResolvedAdjacency.java | 94 +++++++++++
.../org/apache/graphar/core/ChunkRangeTest.java | 75 +++++++++
.../core/OrderedAdjacencyResolverFixtureTest.java | 179 +++++++++++++++++++++
maven-projects/pom.xml | 1 +
12 files changed, 1040 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/java-sdk.yml b/.github/workflows/java-sdk.yml
index a8779ee0..5a45ac49 100644
--- a/.github/workflows/java-sdk.yml
+++ b/.github/workflows/java-sdk.yml
@@ -55,7 +55,7 @@ jobs:
test:
runs-on: ubuntu-latest
env:
- SDK_MODULES: storage-api,storage-local,io-api,storage-s3
+ SDK_MODULES: storage-api,storage-local,io-api,storage-s3,core
steps:
- uses: actions/checkout@v7
with:
diff --git a/maven-projects/core/pom.xml b/maven-projects/core/pom.xml
new file mode 100644
index 00000000..529ae756
--- /dev/null
+++ b/maven-projects/core/pom.xml
@@ -0,0 +1,88 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.graphar</groupId>
+ <artifactId>graphar-root</artifactId>
+ <version>${graphar.version}</version>
+ <relativePath>../pom.xml</relativePath>
+ </parent>
+
+ <artifactId>graphar-core</artifactId>
+ <packaging>jar</packaging>
+ <version>${graphar.version}</version>
+ <name>graphar-core</name>
+
+ <properties>
+ <maven.compiler.source>11</maven.compiler.source>
+ <maven.compiler.target>11</maven.compiler.target>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.graphar</groupId>
+ <artifactId>graphar-info</artifactId>
+ <version>${graphar.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>4.13.2</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>com.diffplug.spotless</groupId>
+ <artifactId>spotless-maven-plugin</artifactId>
+ <version>${spotless-maven-plugin.version}</version>
+ <configuration>
+ <java>
+ <googleJavaFormat>
+ <version>1.7</version>
+ <style>AOSP</style>
+ </googleJavaFormat>
+ </java>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-javadocs</id>
+ <goals>
+ <goal>jar</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
diff --git
a/maven-projects/core/src/main/java/org/apache/graphar/core/ChunkMath.java
b/maven-projects/core/src/main/java/org/apache/graphar/core/ChunkMath.java
new file mode 100644
index 00000000..0fca2b2c
--- /dev/null
+++ b/maven-projects/core/src/main/java/org/apache/graphar/core/ChunkMath.java
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.graphar.core;
+
+/** Long-safe operations on GraphAr vertex and edge chunks. */
+public final class ChunkMath {
+ private ChunkMath() {}
+
+ /**
+ * Returns the chunk containing a non-negative element identifier.
+ *
+ * @param elementId an element identifier
+ * @param chunkSize a positive chunk size
+ * @return the zero-based chunk index
+ */
+ public static long chunkIndex(long elementId, long chunkSize) {
+ validateElementId(elementId);
+ validateChunkSize(chunkSize);
+ return elementId / chunkSize;
+ }
+
+ /**
+ * Returns an element's zero-based position inside its chunk.
+ *
+ * @param elementId an element identifier
+ * @param chunkSize a positive chunk size
+ * @return the zero-based offset within the chunk
+ */
+ public static long offsetInChunk(long elementId, long chunkSize) {
+ validateElementId(elementId);
+ validateChunkSize(chunkSize);
+ return elementId % chunkSize;
+ }
+
+ /**
+ * Returns the number of chunks required for a non-negative number of
elements.
+ *
+ * @param elementCount a number of elements
+ * @param chunkSize a positive chunk size
+ * @return the number of chunks needed to contain the elements
+ */
+ public static long chunkCount(long elementCount, long chunkSize) {
+ if (elementCount < 0) {
+ throw new IllegalArgumentException(
+ "Element count must be non-negative: " + elementCount);
+ }
+ validateChunkSize(chunkSize);
+ return elementCount == 0 ? 0 : 1 + (elementCount - 1) / chunkSize;
+ }
+
+ static void validateElementId(long elementId) {
+ if (elementId < 0) {
+ throw new IllegalArgumentException("Element ID must be
non-negative: " + elementId);
+ }
+ }
+
+ static void validateChunkSize(long chunkSize) {
+ if (chunkSize <= 0) {
+ throw new IllegalArgumentException("Chunk size must be positive: "
+ chunkSize);
+ }
+ }
+}
diff --git
a/maven-projects/core/src/main/java/org/apache/graphar/core/ChunkRange.java
b/maven-projects/core/src/main/java/org/apache/graphar/core/ChunkRange.java
new file mode 100644
index 00000000..982946c0
--- /dev/null
+++ b/maven-projects/core/src/main/java/org/apache/graphar/core/ChunkRange.java
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.graphar.core;
+
+import java.util.Objects;
+
+/** A half-open range of non-negative chunk indexes. */
+public final class ChunkRange {
+ private final long begin;
+ private final long end;
+
+ /**
+ * Creates a chunk range.
+ *
+ * @param begin the first included non-negative chunk index
+ * @param end the first excluded chunk index
+ */
+ public ChunkRange(long begin, long end) {
+ if (begin < 0) {
+ throw new IllegalArgumentException("Chunk range begin must be
non-negative: " + begin);
+ }
+ if (end < begin) {
+ throw new IllegalArgumentException(
+ "Chunk range end must not precede begin: [" + begin + ", "
+ end + ")");
+ }
+ this.begin = begin;
+ this.end = end;
+ }
+
+ /**
+ * Returns the first included chunk index.
+ *
+ * @return the first included chunk index
+ */
+ public long begin() {
+ return begin;
+ }
+
+ /**
+ * Returns the first excluded chunk index.
+ *
+ * @return the first excluded chunk index
+ */
+ public long end() {
+ return end;
+ }
+
+ /**
+ * Returns whether this range selects no chunks.
+ *
+ * @return whether the range is empty
+ */
+ public boolean isEmpty() {
+ return begin == end;
+ }
+
+ /**
+ * Returns whether {@code chunkIndex} belongs to this range.
+ *
+ * @param chunkIndex a chunk index
+ * @return whether the chunk index belongs to this range
+ */
+ public boolean contains(long chunkIndex) {
+ return chunkIndex >= begin && chunkIndex < end;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (this == other) {
+ return true;
+ }
+ if (!(other instanceof ChunkRange)) {
+ return false;
+ }
+ ChunkRange that = (ChunkRange) other;
+ return begin == that.begin && end == that.end;
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(begin, end);
+ }
+
+ @Override
+ public String toString() {
+ return "ChunkRange[" + begin + ", " + end + ")";
+ }
+}
diff --git
a/maven-projects/core/src/main/java/org/apache/graphar/core/EdgeRange.java
b/maven-projects/core/src/main/java/org/apache/graphar/core/EdgeRange.java
new file mode 100644
index 00000000..ab596a98
--- /dev/null
+++ b/maven-projects/core/src/main/java/org/apache/graphar/core/EdgeRange.java
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.graphar.core;
+
+import java.util.Objects;
+
+/** A validated half-open range of edge rows within one GraphAr vertex
partition. */
+public final class EdgeRange {
+ private final long begin;
+ private final long end;
+
+ private EdgeRange(long begin, long end) {
+ this.begin = begin;
+ this.end = end;
+ }
+
+ /**
+ * Creates an edge range from the two adjacent values of an ordered offset
table.
+ *
+ * @param begin the first included edge row
+ * @param end the first excluded edge row
+ * @return the validated half-open edge range
+ */
+ public static EdgeRange fromOffsets(long begin, long end) {
+ if (begin < 0) {
+ throw new IllegalArgumentException("Edge range begin must be
non-negative: " + begin);
+ }
+ if (end < begin) {
+ throw new IllegalArgumentException(
+ "Offset values must be monotonic: begin=" + begin + ",
end=" + end);
+ }
+ return new EdgeRange(begin, end);
+ }
+
+ /**
+ * Returns the first included edge row.
+ *
+ * @return the first included edge row
+ */
+ public long begin() {
+ return begin;
+ }
+
+ /**
+ * Returns the first excluded edge row.
+ *
+ * @return the first excluded edge row
+ */
+ public long end() {
+ return end;
+ }
+
+ /**
+ * Returns the number of selected edge rows.
+ *
+ * @return the number of selected edge rows
+ */
+ public long length() {
+ return end - begin;
+ }
+
+ /**
+ * Returns whether this range selects no edge rows.
+ *
+ * @return whether the range is empty
+ */
+ public boolean isEmpty() {
+ return begin == end;
+ }
+
+ /**
+ * Returns the half-open range of edge chunks intersecting this edge range.
+ *
+ * @param edgeChunkSize a positive number of edge rows per chunk
+ * @return the chunk range intersecting this edge range
+ */
+ public ChunkRange edgeChunks(long edgeChunkSize) {
+ long first = ChunkMath.chunkIndex(begin, edgeChunkSize);
+ if (isEmpty()) {
+ return new ChunkRange(first, first);
+ }
+ return new ChunkRange(first, ChunkMath.chunkCount(end, edgeChunkSize));
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (this == other) {
+ return true;
+ }
+ if (!(other instanceof EdgeRange)) {
+ return false;
+ }
+ EdgeRange that = (EdgeRange) other;
+ return begin == that.begin && end == that.end;
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(begin, end);
+ }
+
+ @Override
+ public String toString() {
+ return "EdgeRange[" + begin + ", " + end + ")";
+ }
+}
diff --git
a/maven-projects/core/src/main/java/org/apache/graphar/core/OffsetChunk.java
b/maven-projects/core/src/main/java/org/apache/graphar/core/OffsetChunk.java
new file mode 100644
index 00000000..81932444
--- /dev/null
+++ b/maven-projects/core/src/main/java/org/apache/graphar/core/OffsetChunk.java
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.graphar.core;
+
+import java.util.Arrays;
+import java.util.Objects;
+
+/** A validated ordered-layout offset chunk with one more value than local
vertices. */
+public final class OffsetChunk {
+ private final long vertexChunkIndex;
+ private final long[] offsets;
+
+ private OffsetChunk(long vertexChunkIndex, long[] offsets) {
+ this.vertexChunkIndex = vertexChunkIndex;
+ this.offsets = offsets;
+ }
+
+ /**
+ * Creates an immutable offset chunk read from the offset file of {@code
vertexChunkIndex}.
+ * Values must start at zero and be non-negative and monotonic.
+ */
+ public static OffsetChunk of(long vertexChunkIndex, long[] offsets) {
+ if (vertexChunkIndex < 0) {
+ throw new IllegalArgumentException(
+ "Vertex chunk index must be non-negative: " +
vertexChunkIndex);
+ }
+ Objects.requireNonNull(offsets, "Offset values cannot be null.");
+ if (offsets.length < 2) {
+ throw new IllegalArgumentException("An offset chunk must contain
at least two values.");
+ }
+ long[] copy = Arrays.copyOf(offsets, offsets.length);
+ if (copy[0] != 0) {
+ throw new IllegalArgumentException("The first offset value must be
zero: " + copy[0]);
+ }
+ long previous = copy[0];
+ for (int index = 1; index < copy.length; index++) {
+ long current = copy[index];
+ if (current < previous) {
+ throw new IllegalArgumentException(
+ "Offset values must be monotonic at index "
+ + index
+ + ": "
+ + previous
+ + " > "
+ + current);
+ }
+ previous = current;
+ }
+ return new OffsetChunk(vertexChunkIndex, copy);
+ }
+
+ /** Returns the vertex chunk this offset chunk was read from. */
+ public long vertexChunkIndex() {
+ return vertexChunkIndex;
+ }
+
+ /** Returns the number of local vertices represented by this chunk. */
+ public long vertexCount() {
+ return offsets.length - 1L;
+ }
+
+ /** Resolves the half-open edge range for one local vertex position. */
+ public EdgeRange rangeFor(long localVertexIndex) {
+ if (localVertexIndex < 0 || localVertexIndex >= vertexCount()) {
+ throw new IllegalArgumentException(
+ "Local vertex index must be in [0, "
+ + vertexCount()
+ + "): "
+ + localVertexIndex);
+ }
+ int index = Math.toIntExact(localVertexIndex);
+ return EdgeRange.fromOffsets(offsets[index], offsets[index + 1]);
+ }
+
+ /** Fails if the final offset does not equal the corresponding partition
edge count. */
+ public void validateEdgeCount(long edgeCount) {
+ if (edgeCount < 0) {
+ throw new IllegalArgumentException("Edge count must be
non-negative: " + edgeCount);
+ }
+ long finalOffset = offsets[offsets.length - 1];
+ if (finalOffset != edgeCount) {
+ throw new IllegalArgumentException(
+ "Final offset must equal edge count: " + finalOffset + "
!= " + edgeCount);
+ }
+ }
+
+ @Override
+ public String toString() {
+ return "OffsetChunk{vertexChunk="
+ + vertexChunkIndex
+ + ", vertexCount="
+ + vertexCount()
+ + ", edgeCount="
+ + offsets[offsets.length - 1]
+ + "}";
+ }
+}
diff --git
a/maven-projects/core/src/main/java/org/apache/graphar/core/OffsetLocation.java
b/maven-projects/core/src/main/java/org/apache/graphar/core/OffsetLocation.java
new file mode 100644
index 00000000..ec2a438d
--- /dev/null
+++
b/maven-projects/core/src/main/java/org/apache/graphar/core/OffsetLocation.java
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.graphar.core;
+
+import java.net.URI;
+import java.util.Objects;
+
+/** The offset chunk and pair index required to resolve one ordered-layout
vertex. */
+public final class OffsetLocation {
+ private final long vertexId;
+ private final long vertexChunkIndex;
+ private final long offsetIndex;
+ private final URI offsetChunkUri;
+
+ OffsetLocation(long vertexId, long vertexChunkIndex, long offsetIndex, URI
offsetChunkUri) {
+ this.vertexId = vertexId;
+ this.vertexChunkIndex = vertexChunkIndex;
+ this.offsetIndex = offsetIndex;
+ this.offsetChunkUri =
+ Objects.requireNonNull(offsetChunkUri, "Offset chunk URI
cannot be null.");
+ }
+
+ public long vertexId() {
+ return vertexId;
+ }
+
+ public long vertexChunkIndex() {
+ return vertexChunkIndex;
+ }
+
+ /** Returns the first of the two adjacent offset values to read. */
+ public long offsetIndex() {
+ return offsetIndex;
+ }
+
+ public URI offsetChunkUri() {
+ return offsetChunkUri;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (this == other) {
+ return true;
+ }
+ if (!(other instanceof OffsetLocation)) {
+ return false;
+ }
+ OffsetLocation that = (OffsetLocation) other;
+ return vertexId == that.vertexId
+ && vertexChunkIndex == that.vertexChunkIndex
+ && offsetIndex == that.offsetIndex
+ && offsetChunkUri.equals(that.offsetChunkUri);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(vertexId, vertexChunkIndex, offsetIndex,
offsetChunkUri);
+ }
+
+ @Override
+ public String toString() {
+ return "OffsetLocation{vertexId="
+ + vertexId
+ + ", vertexChunk="
+ + vertexChunkIndex
+ + ", offsetIndex="
+ + offsetIndex
+ + ", offsetChunkUri="
+ + offsetChunkUri
+ + "}";
+ }
+}
diff --git
a/maven-projects/core/src/main/java/org/apache/graphar/core/OrderedAdjacencyResolver.java
b/maven-projects/core/src/main/java/org/apache/graphar/core/OrderedAdjacencyResolver.java
new file mode 100644
index 00000000..6de547cc
--- /dev/null
+++
b/maven-projects/core/src/main/java/org/apache/graphar/core/OrderedAdjacencyResolver.java
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.graphar.core;
+
+import java.net.URI;
+import java.util.Objects;
+import org.apache.graphar.info.EdgeInfo;
+import org.apache.graphar.info.type.AdjListType;
+
+/** Resolves GraphAr ordered adjacency metadata into offset and edge-chunk
locations. */
+public final class OrderedAdjacencyResolver {
+ private final EdgeInfo edgeInfo;
+ private final AdjListType adjListType;
+ private final long vertexChunkSize;
+
+ public OrderedAdjacencyResolver(EdgeInfo edgeInfo, AdjListType
adjListType) {
+ this.edgeInfo = Objects.requireNonNull(edgeInfo, "Edge info cannot be
null.");
+ this.adjListType =
+ Objects.requireNonNull(adjListType, "Adjacency list type
cannot be null.");
+ if (!adjListType.isOrdered()) {
+ throw new IllegalArgumentException(
+ "An ordered adjacency resolver requires an ordered layout:
" + adjListType);
+ }
+ if (!edgeInfo.hasAdjListType(adjListType)) {
+ throw new IllegalArgumentException(
+ "Edge info does not declare adjacency layout: " +
adjListType);
+ }
+ this.vertexChunkSize =
+ adjListType == AdjListType.ordered_by_source
+ ? edgeInfo.getSrcChunkSize()
+ : edgeInfo.getDstChunkSize();
+ ChunkMath.validateChunkSize(vertexChunkSize);
+ ChunkMath.validateChunkSize(edgeInfo.getChunkSize());
+ }
+
+ /** Locates the offset pair that the physical reader must fetch for {@code
vertexId}. */
+ public OffsetLocation locate(long vertexId) {
+ long vertexChunkIndex = ChunkMath.chunkIndex(vertexId,
vertexChunkSize);
+ long offsetIndex = ChunkMath.offsetInChunk(vertexId, vertexChunkSize);
+ URI offsetChunkUri = edgeInfo.getOffsetChunkUri(adjListType,
vertexChunkIndex);
+ return new OffsetLocation(vertexId, vertexChunkIndex, offsetIndex,
offsetChunkUri);
+ }
+
+ /** Combines a vertex location and its two ordered-offset values into
exact edge chunks. */
+ public ResolvedAdjacency resolve(long vertexId, long offsetBegin, long
offsetEnd) {
+ OffsetLocation offsetLocation = locate(vertexId);
+ EdgeRange edgeRange = EdgeRange.fromOffsets(offsetBegin, offsetEnd);
+ return resolved(offsetLocation, edgeRange);
+ }
+
+ /** Resolves a vertex using a complete, validated offset chunk read by a
physical backend. */
+ public ResolvedAdjacency resolve(long vertexId, OffsetChunk offsetChunk) {
+ Objects.requireNonNull(offsetChunk, "Offset chunk cannot be null.");
+ OffsetLocation offsetLocation = locate(vertexId);
+ if (offsetChunk.vertexChunkIndex() !=
offsetLocation.vertexChunkIndex()) {
+ throw new IllegalArgumentException(
+ "Offset chunk "
+ + offsetChunk.vertexChunkIndex()
+ + " does not hold vertex "
+ + vertexId
+ + ", which lives in vertex chunk "
+ + offsetLocation.vertexChunkIndex());
+ }
+ return resolved(offsetLocation,
offsetChunk.rangeFor(offsetLocation.offsetIndex()));
+ }
+
+ private ResolvedAdjacency resolved(OffsetLocation offsetLocation,
EdgeRange edgeRange) {
+ return new ResolvedAdjacency(
+ edgeInfo,
+ adjListType,
+ offsetLocation,
+ edgeRange,
+ edgeRange.edgeChunks(edgeInfo.getChunkSize()));
+ }
+}
diff --git
a/maven-projects/core/src/main/java/org/apache/graphar/core/ResolvedAdjacency.java
b/maven-projects/core/src/main/java/org/apache/graphar/core/ResolvedAdjacency.java
new file mode 100644
index 00000000..7d729c05
--- /dev/null
+++
b/maven-projects/core/src/main/java/org/apache/graphar/core/ResolvedAdjacency.java
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.graphar.core;
+
+import java.net.URI;
+import java.util.Objects;
+import org.apache.graphar.info.EdgeInfo;
+import org.apache.graphar.info.type.AdjListType;
+
+/** The physical GraphAr chunks selected by one ordered-layout vertex's offset
range. */
+public final class ResolvedAdjacency {
+ private final EdgeInfo edgeInfo;
+ private final AdjListType adjListType;
+ private final OffsetLocation offsetLocation;
+ private final EdgeRange edgeRange;
+ private final ChunkRange edgeChunks;
+
+ ResolvedAdjacency(
+ EdgeInfo edgeInfo,
+ AdjListType adjListType,
+ OffsetLocation offsetLocation,
+ EdgeRange edgeRange,
+ ChunkRange edgeChunks) {
+ this.edgeInfo = Objects.requireNonNull(edgeInfo, "Edge info cannot be
null.");
+ this.adjListType =
+ Objects.requireNonNull(adjListType, "Adjacency list type
cannot be null.");
+ this.offsetLocation =
+ Objects.requireNonNull(offsetLocation, "Offset location cannot
be null.");
+ this.edgeRange = Objects.requireNonNull(edgeRange, "Edge range cannot
be null.");
+ this.edgeChunks = Objects.requireNonNull(edgeChunks, "Edge chunk range
cannot be null.");
+ }
+
+ public OffsetLocation offsetLocation() {
+ return offsetLocation;
+ }
+
+ public EdgeRange edgeRange() {
+ return edgeRange;
+ }
+
+ public ChunkRange edgeChunks() {
+ return edgeChunks;
+ }
+
+ public URI edgeCountUri() {
+ return edgeInfo.getEdgesNumFileUri(adjListType,
offsetLocation.vertexChunkIndex());
+ }
+
+ /** Returns an adjacency URI only for an edge chunk selected by this
result. */
+ public URI adjacencyChunkUri(long edgeChunkIndex) {
+ if (!edgeChunks.contains(edgeChunkIndex)) {
+ throw new IllegalArgumentException(
+ "Edge chunk "
+ + edgeChunkIndex
+ + " is outside the resolved range ["
+ + edgeChunks.begin()
+ + ", "
+ + edgeChunks.end()
+ + ")");
+ }
+ return edgeInfo.getAdjacentListChunkUri(
+ adjListType, offsetLocation.vertexChunkIndex(),
edgeChunkIndex);
+ }
+
+ @Override
+ public String toString() {
+ return "ResolvedAdjacency{adjListType="
+ + adjListType
+ + ", offsetLocation="
+ + offsetLocation
+ + ", edgeRange="
+ + edgeRange
+ + ", edgeChunks="
+ + edgeChunks
+ + "}";
+ }
+}
diff --git
a/maven-projects/core/src/test/java/org/apache/graphar/core/ChunkRangeTest.java
b/maven-projects/core/src/test/java/org/apache/graphar/core/ChunkRangeTest.java
new file mode 100644
index 00000000..050c58d0
--- /dev/null
+++
b/maven-projects/core/src/test/java/org/apache/graphar/core/ChunkRangeTest.java
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.graphar.core;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+
+import org.junit.Test;
+
+/** Tests GraphAr half-open chunk and edge range arithmetic. */
+public class ChunkRangeTest {
+ @Test
+ public void derivesChunkPositionsAndCountsWithoutOverflow() {
+ assertEquals(2L, ChunkMath.chunkIndex(299L, 100L));
+ assertEquals(99L, ChunkMath.offsetInChunk(299L, 100L));
+ assertEquals(0L, ChunkMath.chunkCount(0L, 100L));
+ assertEquals(3L, ChunkMath.chunkCount(201L, 100L));
+ assertEquals(Long.MAX_VALUE, ChunkMath.chunkCount(Long.MAX_VALUE, 1L));
+ }
+
+ @Test
+ public void choosesTheChunkContainingTheLastIncludedEdge() {
+ EdgeRange range = EdgeRange.fromOffsets(1008L, 1061L);
+
+ ChunkRange chunks = range.edgeChunks(1024L);
+
+ assertEquals(0L, chunks.begin());
+ assertEquals(2L, chunks.end());
+ assertTrue(chunks.contains(0L));
+ assertTrue(chunks.contains(1L));
+ assertFalse(chunks.contains(2L));
+ }
+
+ @Test
+ public void keepsEmptyRangesAtTheirBeginChunk() {
+ ChunkRange chunks = EdgeRange.fromOffsets(2048L,
2048L).edgeChunks(1024L);
+
+ assertEquals(2L, chunks.begin());
+ assertEquals(2L, chunks.end());
+ assertTrue(chunks.isEmpty());
+ }
+
+ @Test(expected = IllegalArgumentException.class)
+ public void rejectsNegativeElementIdentifiers() {
+ ChunkMath.chunkIndex(-1L, 1L);
+ }
+
+ @Test(expected = IllegalArgumentException.class)
+ public void rejectsNonPositiveChunkSizes() {
+ EdgeRange.fromOffsets(0L, 1L).edgeChunks(0L);
+ }
+
+ @Test(expected = IllegalArgumentException.class)
+ public void rejectsDecreasingOffsetPairs() {
+ EdgeRange.fromOffsets(2L, 1L);
+ }
+}
diff --git
a/maven-projects/core/src/test/java/org/apache/graphar/core/OrderedAdjacencyResolverFixtureTest.java
b/maven-projects/core/src/test/java/org/apache/graphar/core/OrderedAdjacencyResolverFixtureTest.java
new file mode 100644
index 00000000..863a774a
--- /dev/null
+++
b/maven-projects/core/src/test/java/org/apache/graphar/core/OrderedAdjacencyResolverFixtureTest.java
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.graphar.core;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertThrows;
+import static org.junit.Assert.assertTrue;
+
+import java.net.URI;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import org.apache.graphar.info.EdgeInfo;
+import
org.apache.graphar.info.loader.impl.LocalFileSystemStringGraphInfoLoader;
+import org.apache.graphar.info.type.AdjListType;
+import org.junit.Assume;
+import org.junit.Test;
+
+/** Verifies ordered adjacency resolution against the canonical LDBC GraphAr
fixture. */
+public class OrderedAdjacencyResolverFixtureTest {
+ @Test
+ public void resolvesLdbcOffsetPairAcrossTwoCanonicalAdjacencyChunks()
throws Exception {
+ Path fixtureRoot = canonicalFixtureRoot();
+ Assume.assumeTrue(
+ "The canonical GraphAr testing fixtures are unavailable."
+ + " Set GAR_TEST_DATA to the testing directory to run
this test.",
+ fixtureRoot != null);
+ EdgeInfo edgeInfo =
+ new LocalFileSystemStringGraphInfoLoader()
+
.loadEdgeInfo(fixtureRoot.resolve("person_knows_person.edge.yml").toUri());
+ OrderedAdjacencyResolver resolver =
+ new OrderedAdjacencyResolver(edgeInfo,
AdjListType.ordered_by_source);
+ OffsetChunk offsets = OffsetChunk.of(2,
offsetsForCanonicalVertex297());
+
+ assertEquals(100, offsets.vertexCount());
+ offsets.validateEdgeCount(1077);
+
+ ResolvedAdjacency resolved = resolver.resolve(297, offsets);
+
+ assertEquals(
+ new OffsetLocation(
+ 297,
+ 2,
+ 97,
+
URI.create("edge/person_knows_person/ordered_by_source/offset/chunk2")),
+ resolved.offsetLocation());
+ assertEquals(EdgeRange.fromOffsets(1008, 1061), resolved.edgeRange());
+ assertEquals(new ChunkRange(0, 2), resolved.edgeChunks());
+ assertEquals(
+
URI.create("edge/person_knows_person/ordered_by_source/adj_list/part2/chunk0"),
+ resolved.adjacencyChunkUri(0));
+ assertEquals(
+
URI.create("edge/person_knows_person/ordered_by_source/adj_list/part2/chunk1"),
+ resolved.adjacencyChunkUri(1));
+ assertEquals(
+
URI.create("edge/person_knows_person/ordered_by_source/edge_count2"),
+ resolved.edgeCountUri());
+ assertFalse(resolved.edgeChunks().isEmpty());
+ assertTrue(
+ Files.isRegularFile(
+ fixtureRoot.resolve(
+
"edge/person_knows_person/ordered_by_source/adj_list/part2/chunk0")));
+ assertTrue(
+ Files.isRegularFile(
+ fixtureRoot.resolve(
+
"edge/person_knows_person/ordered_by_source/adj_list/part2/chunk1")));
+ }
+
+ @Test
+ public void preservesLongAndHalfOpenBoundaries() {
+ assertEquals(2, ChunkMath.chunkIndex(299, 100));
+ assertEquals(99, ChunkMath.offsetInChunk(299, 100));
+ assertEquals(3, ChunkMath.chunkCount(201, 100));
+ assertEquals(Long.MAX_VALUE, ChunkMath.chunkCount(Long.MAX_VALUE, 1));
+
+ EdgeRange range = EdgeRange.fromOffsets(1024, 1025);
+ assertEquals(1, range.edgeChunks(1024).begin());
+ assertEquals(2, range.edgeChunks(1024).end());
+ assertFalse(range.edgeChunks(1024).isEmpty());
+ assertEquals(1, EdgeRange.fromOffsets(1024,
1024).edgeChunks(1024).begin());
+ assertEquals(1, EdgeRange.fromOffsets(1024,
1024).edgeChunks(1024).end());
+
+ assertThrows(IllegalArgumentException.class, () ->
ChunkMath.chunkIndex(-1, 1));
+ assertThrows(IllegalArgumentException.class, () ->
EdgeRange.fromOffsets(5, 4));
+ assertThrows(IllegalArgumentException.class, () -> OffsetChunk.of(0,
new long[] {1, 1}));
+ assertThrows(IllegalArgumentException.class, () -> OffsetChunk.of(0,
new long[] {0, 2, 1}));
+ assertThrows(IllegalArgumentException.class, () -> OffsetChunk.of(-1,
new long[] {0, 1}));
+ }
+
+ @Test
+ public void rejectsAnOffsetChunkReadFromAnotherVertexChunk() throws
Exception {
+ Path fixtureRoot = canonicalFixtureRoot();
+ Assume.assumeTrue(
+ "The canonical GraphAr testing fixtures are unavailable."
+ + " Set GAR_TEST_DATA to the testing directory to run
this test.",
+ fixtureRoot != null);
+ EdgeInfo edgeInfo =
+ new LocalFileSystemStringGraphInfoLoader()
+
.loadEdgeInfo(fixtureRoot.resolve("person_knows_person.edge.yml").toUri());
+ OrderedAdjacencyResolver resolver =
+ new OrderedAdjacencyResolver(edgeInfo,
AdjListType.ordered_by_source);
+ OffsetChunk wrongChunk = OffsetChunk.of(1,
offsetsForCanonicalVertex297());
+
+ IllegalArgumentException failure =
+ assertThrows(
+ IllegalArgumentException.class, () ->
resolver.resolve(297, wrongChunk));
+
+ assertTrue(failure.getMessage(), failure.getMessage().contains("vertex
chunk 2"));
+ }
+
+ @Test
+ public void describesRangesAsValues() {
+ assertEquals(new ChunkRange(0, 2), new ChunkRange(0, 2));
+ assertEquals(new ChunkRange(0, 2).hashCode(), new ChunkRange(0,
2).hashCode());
+ assertNotEquals(new ChunkRange(0, 2), new ChunkRange(0, 3));
+ assertEquals(EdgeRange.fromOffsets(3, 7), EdgeRange.fromOffsets(3, 7));
+ assertNotEquals(EdgeRange.fromOffsets(3, 7), EdgeRange.fromOffsets(3,
8));
+ assertEquals("ChunkRange[0, 2)", new ChunkRange(0, 2).toString());
+ assertEquals("EdgeRange[3, 7)", EdgeRange.fromOffsets(3,
7).toString());
+ }
+
+ /**
+ * Returns the parquet LDBC fixture directory, or {@code null} when the
canonical testing data
+ * is unavailable. Resolution follows the same order as the metadata
module tests: the {@code
+ * GAR_TEST_DATA} environment variable, the {@code gar.test.data} system
property, then the
+ * testing directory of a full checkout.
+ */
+ private static Path canonicalFixtureRoot() {
+ String configured = System.getenv("GAR_TEST_DATA");
+ if (configured == null) {
+ configured = System.getProperty("gar.test.data");
+ }
+ if (configured != null) {
+ Path candidate = Path.of(configured, "ldbc_sample", "parquet");
+ return isFixtureRoot(candidate) ? candidate : null;
+ }
+ for (String relative : new String[] {"../../testing", "../testing",
"testing"}) {
+ Path candidate = Path.of(relative, "ldbc_sample", "parquet");
+ if (isFixtureRoot(candidate)) {
+ return candidate;
+ }
+ }
+ return null;
+ }
+
+ private static boolean isFixtureRoot(Path candidate) {
+ return
Files.isRegularFile(candidate.resolve("person_knows_person.edge.yml"));
+ }
+
+ private static long[] offsetsForCanonicalVertex297() {
+ long[] offsets = new long[101];
+ for (int index = 1; index < 97; index++) {
+ offsets[index] = index * 10L;
+ }
+ offsets[97] = 1008;
+ offsets[98] = 1061;
+ offsets[99] = 1061;
+ offsets[100] = 1077;
+ return offsets;
+ }
+}
diff --git a/maven-projects/pom.xml b/maven-projects/pom.xml
index 5ea0b89c..1ccf661b 100644
--- a/maven-projects/pom.xml
+++ b/maven-projects/pom.xml
@@ -82,6 +82,7 @@
<module>storage-local</module>
<module>io-api</module>
<module>storage-s3</module>
+ <module>core</module>
</modules>
<build>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
