leaves12138 commented on code in PR #822: URL: https://github.com/apache/paimon-rust/pull/822#discussion_r3999895754
########## crates/paimon/src/table/vector_read.rs: ########## @@ -0,0 +1,117 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +//! Vector index execution contract and the shared scan -> plan -> read pipeline. + +use crate::spec::{CoreOptions, Predicate}; +use crate::table::de_vector_read::DeVectorRead; +use crate::table::de_vector_scan::PreparedVectorSearchFilter; +use crate::table::pk_vector_read::PkVectorRead; +use crate::table::pk_vector_search_params::PkVectorSearchParams; +use crate::table::vector_scan::{PlanContext, VectorScanPlan, VectorScanWork}; +use crate::table::vector_search_common::{take_only_result, targets_primary_key_column}; +use crate::table::Table; +use crate::vector_search::SearchResult; +use roaring::RoaringTreemap; +use std::collections::HashMap; +use std::future::Future; +use std::sync::Arc; + +/// Execute a resolved plan without resolving another snapshot or manifest. +/// +/// Both implementations return snapshot-scoped search results. The associated +/// plan type prevents passing a DE plan to a PK reader or vice versa. +pub(super) trait Read: Sync { + type Plan; + + fn read( + &self, + plan: Self::Plan, + ) -> impl Future<Output = crate::Result<Vec<SearchResult>>> + Send; +} + +/// Searches a common plan with one query, without replanning or materializing rows. +pub struct VectorRead { + pub(super) batch: BatchVectorRead, +} + +impl VectorRead { + pub async fn read(&self, plan: VectorScanPlan) -> crate::Result<SearchResult> { + take_only_result(self.batch.read(plan).await?, "vector search") + } +} + +/// Searches a common plan with multiple queries, preserving input order and arity. +/// Owns its configuration so it can outlive the builder that created it. +pub struct BatchVectorRead { + context: PlanContext, + reader: VectorReadKind, +} + +enum VectorReadKind { + DataEvolution(DeVectorRead), + PrimaryKey(PkVectorRead), +} Review Comment: [P2] Fix the new Clippy errors so the check job can pass The current head fails the hosted `check` job in Clippy, not because of an intermittent runner failure. I reproduced the same five diagnostics locally with `cargo clippy -p paimon --offline --features fulltext --lib -- -D warnings`: - `large_enum_variant` here on `VectorReadKind`, and on `VectorScanWork` (`vector_scan.rs:50`) and `VectorScanKind` (`vector_scan.rs:129`). For example, this enum embeds a 1,232-byte PK reader alongside a 24-byte DE reader. - `too_many_arguments` on `BatchVectorRead::new` (`vector_read.rs:71`, 8 arguments). - `needless_borrow` on `table: Some(&pinned_table)` (`de_vector_read.rs:228`); `pinned_table` is already a reference. Please address the enum layout warnings (for example by boxing the large variants), remove the redundant borrow, and refactor or narrowly annotate the constructor argument count. Then rerun the workspace Clippy command used by CI. These are errors under the existing `-D warnings` policy, so passing the runtime tests alone does not make this head pass CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
