YannByron opened a new issue, #1402:
URL: https://github.com/apache/paimon-rust/issues/1402

   ### Search before asking
   
   This is a follow-up to #901, which added managed BLOB preselection for Data 
Evolution reads with `LIMIT`. The same path is still disabled when the query 
has no limit. It is separate from the primary-key BLOB issue #1052.
   
   ### What happened
   
   On a Data Evolution append table, a predicate on ordinary columns is applied 
after the format reader materializes managed BLOB payloads when the query has 
no `LIMIT`. A selective query therefore decodes payloads for rows that will be 
discarded.
   
   A deterministic regression can use row IDs 100–103, an updated scalar column 
whose current values are `[4, 2, 3, 1]`, and four managed payloads. Corrupt the 
fourth payload's entry header while keeping its descriptor index readable, then 
project the scalar column and payload with the filter `id > 1` and no limit. 
The excluded fourth payload is decoded and the query fails, although only the 
first three payloads are needed.
   
   ### Expected behavior
   
   For predicates that depend only on ordinary columns, use the current column 
values to select row-ID ranges before materializing managed BLOB payloads, with 
or without a limit. Keep the existing behavior for BLOB-dependent predicates, 
descriptor reads, BLOB views and legacy files without row IDs.
   
   ### Affected version
   
   The `self.limit.is_none()` guard is present on main at 
`8a432a6398ce9e19a57e595b79dcde4d64304bc4`. The regression fixture uses the 
local filesystem and needs no object storage credentials.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to