leaves12138 commented on code in PR #590:
URL: https://github.com/apache/paimon-rust/pull/590#discussion_r3638309737
##########
crates/paimon/src/table/table_scan.rs:
##########
@@ -1224,21 +1657,67 @@ impl<'a> PaimonTableScan<'a> {
data_evolution_read_field_ids: Option<&HashSet<i32>>,
mut trace: Option<&mut ScanTrace>,
) -> crate::Result<Plan> {
+ if matches!(self.limit, Some(0)) {
+ if let Some(trace) = trace {
+ trace.record_final_plan_with_limit(0, 0, 0, 0, true);
+ }
+ return Ok(Plan::new(Vec::new()));
+ }
+ let core_options = CoreOptions::new(self.table.schema().options());
+ let data_evolution_enabled = core_options.data_evolution_enabled();
+ let global_index_settings =
+ self.global_index_scan_settings(&core_options,
data_evolution_enabled)?;
+ let index_entries = self
+ .read_index_manifest_entries(
+ &snapshot,
+ global_index_settings.is_some(),
+ core_options.deletion_vectors_enabled(),
+ )
+ .await?;
+ let manifest_row_ranges = self
+ .manifest_row_ranges(&snapshot, index_entries.as_deref(),
global_index_settings)
+ .await?;
Review Comment:
Could we short-circuit an empty `manifest_row_ranges` here? When the global
index returns `Some(vec![])`, a manifest/file with missing or invalid row-id
metadata makes both row-range pruning helpers fail open. The file then reaches
`split_row_ranges_for_files`, which returns `DataInvalid` because
`row_id_range()` is `None`. As a result, a valid zero-hit Fast-mode lookup on a
legacy data-evolution table fails instead of returning an empty plan. I
reproduced this by planning with `Some(Vec::new())` against a committed file
whose `first_row_id` is `None`; `plan()` returns `Cannot apply selected row
ranges...`. Please return an empty plan before reading/planning data manifests
when the selected range set is empty, and add a regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
