kkzi opened a new issue, #900:
URL: https://github.com/apache/tsfile/issues/900

   ## Description
   
   The C++ `FloatTS2DIFFDecoder::read_batch_float` and 
`DoubleTS2DIFFDecoder::read_batch_double` implementations bypass the 
type-specific scalar decoders and delegate directly to the integer TS_2DIFF 
batch decoders.
   
   FLOAT and DOUBLE TS_2DIFF segments use the Java-compatible layout and 
contain a `maxPointNumber` or overflow-bitmap prefix before the integer delta 
block. The integer batch decoders expect the stream to begin with the TS_2DIFF 
block header, so they interpret the prefix bytes as `write_index` and 
`bit_width`.
   
   ## Symptoms
   
   When a tree reader decodes a FLOAT or DOUBLE measurement using TS_2DIFF:
   
   - the value stream becomes misaligned;
   - the decoder reads invalid block metadata;
   - end-of-input may not clear the decoder's internal remaining state;
   - the tree reader can continue decoding indefinitely instead of completing 
the page.
   
   Scalar `read_float` and `read_double` calls are not affected because they 
consume the segment prefix before decoding the integer delta block.
   
   ## Root cause
   
   The optimized batch overrides reuse 
`TS2DIFFDecoder<int32_t>::read_batch_int32` and 
`TS2DIFFDecoder<int64_t>::read_batch_int64`, then bit-cast the decoded 
integers. This is only valid for the legacy raw C++ layout. It skips the 
scale/overflow handling required by the current FLOAT/DOUBLE segment format.
   
   The affected methods are in `cpp/src/encoding/ts2diff_decoder.h`.
   
   ## Expected behavior
   
   FLOAT/DOUBLE batch decoding must consume each segment prefix and apply 
scale/overflow bitmap handling exactly as scalar decoding does. Batch decoding 
should return the page's declared number of values and terminate with no 
remaining decoder state.
   
   ## Proposed fix
   
   Route FLOAT/DOUBLE batch decoding through the segment-aware 
`read_float`/`read_double` path, and add regression tests that:
   
   1. call `read_batch_float` and `read_batch_double` directly;
   2. cross multiple TS_2DIFF segment boundaries;
   3. cover both normal scale prefixes and overflow bitmap prefixes;
   4. verify decoded values, decoded count, and end-of-input state.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to