YannByron opened a new issue, #1402: URL: https://github.com/apache/paimon-rust/issues/1402
### Search before asking This is a follow-up to #901, which added managed BLOB preselection for Data Evolution reads with `LIMIT`. The same path is still disabled when the query has no limit. It is separate from the primary-key BLOB issue #1052. ### What happened On a Data Evolution append table, a predicate on ordinary columns is applied after the format reader materializes managed BLOB payloads when the query has no `LIMIT`. A selective query therefore decodes payloads for rows that will be discarded. A deterministic regression can use row IDs 100–103, an updated scalar column whose current values are `[4, 2, 3, 1]`, and four managed payloads. Corrupt the fourth payload's entry header while keeping its descriptor index readable, then project the scalar column and payload with the filter `id > 1` and no limit. The excluded fourth payload is decoded and the query fails, although only the first three payloads are needed. ### Expected behavior For predicates that depend only on ordinary columns, use the current column values to select row-ID ranges before materializing managed BLOB payloads, with or without a limit. Keep the existing behavior for BLOB-dependent predicates, descriptor reads, BLOB views and legacy files without row IDs. ### Affected version The `self.limit.is_none()` guard is present on main at `8a432a6398ce9e19a57e595b79dcde4d64304bc4`. The regression fixture uses the local filesystem and needs no object storage credentials. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
