Copilot commented on code in PR #3295:
URL: https://github.com/apache/fluss/pull/3295#discussion_r3214374135
##########
website/docs/apis/java-client.md:
##########
@@ -243,6 +243,62 @@ while (true) {
}
```
+### Batch Scan — Full Primary Key Table
+
+For Primary Key tables, `BatchScanner` reads every live row in the table once
+and stops. The scan is served from a point-in-time RocksDB snapshot on the
+tablet server, so all rows reflect the KV state at the moment the scan was
+opened — concurrent writes are invisible to a running scan.
+
+This is the building block for periodic full-state reads such as:
+
+- **Dashboards / materialised views** — refresh a derived view by re-reading
+ the entire current state of a primary key table on a schedule.
+- **Cache or search-index warm-up** — load every row of a PK table into an
+ external system at startup, with a consistent snapshot guarantee.
+- **Bulk export to OLAP / data lake** — periodic full-snapshot of the table
+ for downstream analytics, complementing the streaming changelog read via
+ `LogScanner`.
+
+`createBatchScanner()` (no arguments) scans the whole table; for partitioned
+tables it expands across all partitions × buckets automatically. Use
+`createBatchScanner(TableBucket)` to scan a single bucket — useful when
+distributing scan work across workers in a parallel engine.
Review Comment:
The documentation claims a full-table batch scan reflects KV state "at the
moment the scan was opened" and that concurrent writes are invisible. In the
current implementation, `createBatchScanner()` builds a `CompositeBatchScanner`
of per-bucket scanners, and each `KvBatchScanner` opens its RocksDB snapshot
lazily on first `pollBatch()`. This means the snapshot boundary is per-bucket
(and may be taken at different times), so concurrent writes could be visible in
buckets that haven't been opened yet. Please clarify this behavior in the docs,
or adjust the implementation to open all bucket scanners up-front if a
table-wide point-in-time view is required.
##########
fluss-client/src/main/java/org/apache/fluss/client/table/scanner/batch/KvBatchScanner.java:
##########
@@ -0,0 +1,345 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.fluss.client.table.scanner.batch;
+
+import org.apache.fluss.annotation.Internal;
+import org.apache.fluss.annotation.VisibleForTesting;
+import org.apache.fluss.client.metadata.MetadataUpdater;
+import org.apache.fluss.exception.LeaderNotAvailableException;
+import org.apache.fluss.metadata.SchemaGetter;
+import org.apache.fluss.metadata.TableBucket;
+import org.apache.fluss.metadata.TableInfo;
+import org.apache.fluss.metadata.TablePath;
+import org.apache.fluss.record.DefaultValueRecordBatch;
+import org.apache.fluss.record.ValueRecord;
+import org.apache.fluss.record.ValueRecordReadContext;
+import org.apache.fluss.row.InternalRow;
+import org.apache.fluss.row.ProjectedRow;
+import org.apache.fluss.rpc.gateway.TabletServerGateway;
+import org.apache.fluss.rpc.messages.PbScanReqForBucket;
+import org.apache.fluss.rpc.messages.ScanKvRequest;
+import org.apache.fluss.rpc.messages.ScanKvResponse;
+import org.apache.fluss.rpc.protocol.Errors;
+import org.apache.fluss.utils.CloseableIterator;
+import org.apache.fluss.utils.SchemaUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nullable;
+import javax.annotation.concurrent.NotThreadSafe;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.time.Duration;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+/**
+ * A {@link BatchScanner} that streams every live row of a single primary-key
bucket from the tablet
+ * server's RocksDB state via a sequence of {@code ScanKv} RPCs. The scan has
snapshot isolation:
+ * rows reflect the KV state at the moment the server opened the snapshot;
concurrent writes after
+ * that point are invisible.
+ *
+ * <p>The next continuation RPC is issued immediately after each response so
the caller's
+ * deserialise/process work overlaps the network round-trip; one in-flight RPC
at a time.
Review Comment:
Spelling: "deserialise" is inconsistent with the rest of the codebase (which
uses "deserialize"). Please update to "deserialize" for consistency.
##########
website/docs/apis/java-client.md:
##########
@@ -243,6 +243,62 @@ while (true) {
}
```
+### Batch Scan — Full Primary Key Table
+
+For Primary Key tables, `BatchScanner` reads every live row in the table once
+and stops. The scan is served from a point-in-time RocksDB snapshot on the
+tablet server, so all rows reflect the KV state at the moment the scan was
+opened — concurrent writes are invisible to a running scan.
+
+This is the building block for periodic full-state reads such as:
+
+- **Dashboards / materialised views** — refresh a derived view by re-reading
Review Comment:
Spelling is inconsistent with the rest of the docs: "materialised" should be
"materialized" (US spelling) to match other pages (e.g.,
website/docs/engine-flink/ddl.md uses "Materialized Table").
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
