u70b3 commented on issue #66497: URL: https://github.com/apache/doris/issues/66497#issuecomment-5212075800
Quick update on the prerequisite track (v3 §12 decision 2 / Q3): the upstream engagement has started — I've filed [lance-format/lance-c#55](https://github.com/lance-format/lance-c/issues/55), an RFC covering all six lance-c extensions, split into three independently-mergeable waves: - **Wave 1 (blocking)**: fragment-scoped uncommitted segment builds with shared-model injection (caller-supplied IVF centroids / PQ codebook) + segment metadata serialization (`pb::IndexMetadata` bytes as the BE→FE transfer format). - **Wave 2 (production quality)**: build progress reporting + cooperative cancellation. - **Wave 3 (symmetry/observability)**: commit-existing-segments C API + per-segment coverage/stats in listing. One finding worth surfacing here, since it de-risks the schedule: the Rust core (9.1.0-beta.3) already implements nearly everything behind the C boundary. `CreateIndexBuilder` already carries `fragments` / `index_uuid` / `train` / `progress` / `transaction_properties` and `execute_uncommitted() -> IndexMetadata` (`rust/lance/src/index/create.rs:50-141`), with scalar fragment-scoped builds supported as well (:270); the injection points `IvfBuildParams.centroids` and `PQBuildParams.codebook` both exist. The only genuine core-side gaps are the cancellation hook and per-segment listing stats — the rest is FFI exposure. This does not presume the Q3 answer: if round-3 prefers a different tracking form, the upstream issue adapts. Round-3 feedback welcome whenever convenient — no urgency imposed by this step. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
