fl0-m opened a new issue, #971:
URL: https://github.com/apache/tsfile/issues/971

   **Version:** apache/tsfile `cpp`, commit 7ca2e79 and `develop` 83fe559 
(2026-09-24). Files written by IoTDB 2.0.11 (table model, aligned, LZ4).
   
   **What happens.** IoTDB's Java writer stores a value page in which a column 
has no value as a single varint `0` 
(`ValueChunkWriter.writeEmptyPageToPageBuffer`), and the Java reader returns an 
empty page as soon as it reads that `0` (`PageHeader.deserializeFrom`: `if 
(uncompressedSize == 0) // Empty Page`). The C++ `PageHeader::deserialize_from` 
(`cpp/src/common/tsfile_common.h`) always reads a second varint (compressed 
size) and, when `deserialize_stat`, a statistic. After an empty page it 
therefore consumes the next page's bytes as this page's header, and every later 
page of that column is misread.
   
   **Minimal reproduction.** In IoTDB, one table with two INT32 fields `a`, 
`b`; write 20,000 rows in tablets of 1,000, `b` filled in the 1st, 3rd, 5th … 
tablet and null in the others; `FLUSH`. Reading `b` (or `a, b`) from the sealed 
file through the C++ reader (e.g. the DuckDB `tsfile` extension) returns **0 
rows**; the truth is 20,000 rows, 10,000 non-null, sum 94,995,000. The same 
data with `b` filled in the first half and null after, or on every other row, 
reads correctly (the empty page must be followed by a non-empty one).
   
   **Fix that works for us** (in `PageHeader::deserialize_from`, after reading 
`uncompressed_size_`):
   ```cpp
   if (uncompressed_size_ == 0) {   // empty page: one varint, no size, no 
statistics
       compressed_size_ = 0;
       statistic_ = nullptr;
       return common::E_OK;
   }
   ```
   The aligned readers already treat `compressed_size_ == 0` as an all-null 
page. With it, 574 of 574 affected column reads over 26 real IoTDB files 
matched the file metadata (0 of 574 before).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to