linliu-code opened a new pull request, #665: URL: https://github.com/apache/hudi-rs/pull/665
**Stacked on #639–#664** — review only the last commit. ## What was wrong The merge-on-read reader took its schema from `hoodie.table.create.schema` — the schema the table was **created** with. Hudi treats that as a last resort. `TableSchemaResolver` resolves in this order: 1. the latest completed commit's metadata (the schema that write actually used) 2. a base file's footer 3. `hoodie.table.create.schema` And this crate already implements the same order in `schema::resolver::resolve_data_schema`. The reader was reaching past both better sources for the weakest one. ## What it does now Reads the base file's own schema — Hudi's tier 2, and what the existing read path effectively uses, so **the two engines now start from the same types**. It is also what the data actually has: under schema evolution the create schema is stale, and the engine evolves each batch to the required schema regardless. ## Three workarounds removed Each of these existed only to make the create schema usable: | Workaround | Why it was needed | |---|---| | Unescaping `\:` | The value arrives as Java writes a properties file, and is not valid JSON until unescaped | | Prepending `_hoodie_*` | The create schema is the user's schema; the files carry meta fields and the merge needs the record key | | Refusing when absent | A table that never recorded one could not be read — **including every reader built from a bare base URI**, which is the shape the cxx bridge uses | The third is the one that mattered: it was a standing gap for the standalone entry point, and it closes here without new plumbing, because every read already has a base file path. ## Base-file-only slices now route through the engine They were held back in #659 because the create schema modelled an Avro map as `Dictionary(Utf8, V)` — an invalid Arrow type — and every parquet fixture here has a map column. The schema no longer comes from there, and the conversion itself is fixed in #664, so the guard is gone. The engine reduces to a base file read for these. The test asserting that setting `v2` does not change such a read now genuinely **compares the two engines** rather than one path against itself. ## Cost One footer request. The engine reads the footer again when it opens the file; collapsing the two is worth doing and is not this change. ## Tests No new ones — this removes special-case handling rather than adding behavior, and the existing coverage tightens as a result: the base-file-only comparison becomes a real differential test, and every harness case that reads a slice now exercises the base-file schema path. Full workspace green: 1180 lib + 79 table-read + 39 datafusion + 21 + 12. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
