yihua opened a new pull request, #987:
URL: https://github.com/apache/hudi-rs/pull/987

   ## Description
   
   A METADATA_ONLY bootstrapped table read its skeleton base files alone, so 
every data column came back NULL; nothing identified a skeleton, and the 
reader's bootstrap refusal was hardcoded off. Full bootstrap merge support is a 
larger piece of work, so this makes those cases fail loudly instead:
   
   - `BaseFile::is_metadata_bootstrap_skeleton` identifies a skeleton by its 
`00000000000001` instant, as Java's 
`AbstractTableFileSystemView#addBootstrapBaseFileIfPresent` does; the bootstrap 
sentinel instants are now named constants in `timeline/instant.rs`.
   - `get_file_slices`, `read` and `read_stream` refuse a planned slice that is 
still on its skeleton, which covers snapshot, time travel, incremental (only 
the slices the window touched), streaming, Python and DataFusion reads. Slices 
rewritten since the bootstrap, and FULL_RECORD bootstrap files, read as before.
   - `validate_for_write` refuses writes to a table with 
`hoodie.bootstrap.base.path` set whose latest slices still include a skeleton, 
before rollback or fencing, so a native write cannot merge NULL data columns 
into a new base file. Ordinary tables skip the check. This conservatively also 
refuses appends on such tables.
   
   Direct per-slice reads (`FileGroupReader::read_file_slice_from_paths`, the 
FFI slice reader) are not checked here.
   
   Closes #920
   
   ## How are the changes test-covered
   
   - [ ] N/A
   - [x] Automated tests (unit and/or integration tests)
   - [ ] Manual tests
     - [ ] Details are described below
   
   New Spark-written COW and MOR fixtures (`v9_metadata_only_bootstrap`, 
bootstrap source and index omitted) drive a read test that returned NULL rows 
on main and now asserts the refusal on every read path, plus correct rows from 
the COW file group rewritten after the bootstrap. A write test asserts upsert 
(COW and MOR) and compaction (MOR) return `Unsupported` and leave the timeline 
unchanged. FULL_RECORD bootstrap tables written by Spark were checked locally 
to still read correctly.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to