sadwitdastreetz commented on code in PR #683:
URL:
https://github.com/apache/incubator-hugegraph-toolchain/pull/683#discussion_r2446442644
##########
hugegraph-loader/src/main/java/org/apache/hugegraph/loader/HugeGraphLoader.java:
##########
@@ -233,15 +733,18 @@ private void loadStruct(InputStruct struct, InputReader
reader) {
ParseTaskBuilder taskBuilder = new ParseTaskBuilder(this.context,
struct);
final int batchSize = this.context.options().batchSize;
List<Line> lines = new ArrayList<>(batchSize);
- for (boolean finished = false; !finished;) {
+ long batchStartTime = System.currentTimeMillis();
+
+ for (boolean finished = false; !finished; ) {
if (this.context.stopped()) {
break;
}
try {
// Read next line from data source
if (reader.hasNext()) {
Line next = reader.next();
Review Comment:
> **Commented Critical Code**: The `reader.confirmOffset()` call is
commented out. This appears critical for progress tracking. If intentional, add
a clear comment explaining why. Without this, data may be reprocessed on
restart or progress reporting could be incorrect.
The reader.confirmOffset() call is not commented out in the current
implementation.
It is invoked after each batch to confirm progress and avoid data
reprocessing on restart.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
