ad1happy2go opened a new pull request, #20523:
URL: https://github.com/apache/hudi/pull/20523

   ### Describe the issue this Pull Request addresses
   
   closes #20522
   
   Snapshot reads in the Trino connector fail on Merge-On-Read tables whose log 
files hold parquet data blocks (`hoodie.logfile.data.block.format=parquet`, 
table version 9 and below), which any Spark record merger writes. The 
file-group reader hands each block to the connector as an `inlinefs://` path, 
and `HudiTrinoReaderContext` rejected it with "Inline log blocks are not 
supported".
   
   ### Summary and Changelog
   
   A parquet data block is a complete parquet file embedded at a byte range of 
the log file, so the connector now reads it in place with the parquet page 
source it already uses for native log files.
   
   - `InlineTrinoInputFile`: a `TrinoInputFile` over one byte range of another 
file, presented as a file starting at position 0. Reads and footer (tail) reads 
stay inside the range.
   - `HudiPageSourceProvider`: log reads with an `inlinefs://` path open 
through that view of the log file.
   - `HudiTrinoReaderContext`: drops the rejection.
   - Tests: `TestInlineTrinoInputFile` covers the range mapping and bounds. 
`TestHudiParquetLogBlocks` reads a version 9 MOR table written with parquet log 
blocks, one record per block so a log file holds several, covering winning and 
losing updates, a soft delete, a hard delete, narrow projections and a filter. 
Its fixture fails if the log files do not hold parquet blocks. Without the fix 
the snapshot tests fail with the original exception.
   
   ### Impact
   
   Trino snapshot queries work on MOR tables with parquet log blocks. No config 
or API change.
   
   ### Risk Level
   
   low. The change only affects reads that previously failed; other log and 
base file reads take the same path as before.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [PR] fix(trino): read inline pa... ad1happy2go (via GitHub) via commits

Reply via email to