mfellner opened a new pull request, #3731:
URL: https://github.com/apache/iceberg-rust/pull/3731

   ## Which issue does this PR close?
   
   - Related to the "Iceberg Cache" epic #1226. No dedicated issue yet; happy 
to file one if preferred.
   
   ## What changes are included in this PR?
   
   `ObjectCache` is crate-private, and `Table::manifest_reader()` always 
bypasses it. A caller that must inspect every manifest itself (for example to 
validate delete manifests before planning) therefore parses each manifest once, 
and `TableScan::plan_files` then fetches and parses it a second time through 
the table's object cache.
   
   This adds one public method:
   
   ```rust
   impl Table {
       pub async fn seed_manifest_cache(&self, manifest_file: &ManifestFile, 
manifest: Arc<Manifest>) -> bool;
   }
   ```
   
   It stores the already parsed manifest under the key 
`ObjectCache::get_manifest` uses (`ManifestKey`: path, snapshot id, sequence 
number, `first_row_id`), so a later `plan_files` over the same `ManifestFile` 
reuses it. The cache is shared by every clone of the table. It returns `false` 
and stores nothing when the cache is disabled, keeps an existing entry, and 
changes nothing when it is not called. The contract documented on the method is 
that `manifest` is the result of reading `manifest_file` with 
`Table::manifest_reader`.
   
   In a Sentinel-style workload, this removes about 2 GETs and an Avro parse 
per manifest per query (measured manifest counts up to ~1,100).
   
   ## Are these changes tested?
   
   Unit tests in `io/object_cache.rs`: a seeded manifest is returned 
(pointer-equal) even after the manifest file is deleted, a second seed keeps 
the first entry, and a disabled cache stores nothing. An Iceberg v3 table 
written by Spark with Iceberg Java 1.11.0 (merge-on-read, deletion vectors) was 
also planned with and without seeding, with the manifest files removed after 
seeding, in an Orcrist fork CI job; that harness is not part of this PR. `cargo 
fmt`, `cargo clippy --all-targets -D warnings` and the object cache tests pass 
locally.
   
   ## AI Disclosure
   
   This change was written with Claude Code (Claude Opus 5.5) and tested by the 
author.
   
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to