bosprimigenious commented on issue #748: URL: https://github.com/apache/hugegraph-toolchain/issues/748#issuecomment-6764354939
AI-generated draft prepared with OpenAI Codex; no maintainer approval or human review is claimed. I propose a small **Rust Loader local-input slice** for design review under #748, before implementation. I have read #772 and SebastianGruza's comment. This excludes their REST fixtures/recorder, Java/Go runners/parity matrix, Loader benchmark datasets/scripts/results, and conditional Rust transport + `/versions`. **Scope:** an offline, single-local-file, uncompressed raw-line source returning original content and a 1-based physical line number. **Non-goals:** CSV/JSON parsing, skip/header policies, property mapping, HTTP, graph writes, checkpoint/resume, or changing Java Loader defaults. At `47c5fc007b0e0418dec99061e9bf0787fdab191b`, `FileLineFetcher.fetch()` calls `BufferedReader.readLine()`, increments physical offset, then handles skip/header and parsing. The proposed seam is only the first two operations: blank/comment lines retain their positions, and physical positions are not committed progress. Java's [documented framing](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/BufferedReader.html#readLine()) accepts LF, CRLF and bare CR. Our local Rust 1.98.1 standard-library probe returned one string for bare-CR-separated input using `BufRead.lines()`; directly wrapping that iterator would break this contract. Java Loader/oracle runtime validation has **not** been completed. **Decisions needed before a child/code PR:** - Is this boundary useful and independent of existing plans? I volunteer for this limited implementation, subject to scope agreement; could a maintainer confirm a mentor/reviewer? - Which workspace location, MSRV/edition and initial platforms should it use? - Is an experimental API restricted to **well-formed UTF-8 within a caller-specified positive record-byte cap** acceptable? Java supports configurable charset, replacement decoding and no line-length cap. Rejecting invalid encoding/oversized records is a proposed boundary requiring review, not default compatibility. **Acceptance:** compare content/physical positions with the pinned Java reference within the agreed subset; test empty/unterminated lines, LF/CRLF/CR across buffers, split UTF-8 characters, exact/over-cap records, mid-read failures and early resource release. Bound buffering to a fixed input buffer plus one capped in-flight record, with synchronous pull/backpressure. No silent truncation/skipping or full-record logging on error. Proposed CPU targets are Linux/macOS, subject to confirmation; require formatting, Clippy, tests and dependency/license checks. Reader-only throughput, first-record latency and peak RSS would use agreed existing inputs, recorded SHA/config/toolchains and raw outputs, coordinated with #772 without duplicating its baseline project. Keep the slice opt-in/read-only, preserving Java commands, configuration and progress formats. Stopping the experimental component requires no data migration. I can document the agreed scope, ownership, compatibility/resource/error decisions and acceptance gates in a focused child RFC before implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
