ColinLeeo opened a new issue, #1034:
URL: https://github.com/apache/tsfile/issues/1034

   <!--
   Licensed to the Apache Software Foundation (ASF) under one
   or more contributor license agreements. See the NOTICE file
   distributed with this work for additional information
   regarding copyright ownership. The ASF licenses this file
   to you under the Apache License, Version 2.0 (the
   "License"); you may not use this file except in compliance
   with the License. You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
   
   Unless required by applicable law or agreed to in writing,
   software distributed under the License is distributed on an
   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   KIND, either express or implied. See the License for the
   specific language governing permissions and limitations
   under the License.
   -->
   
   This is a historical tracking issue for a bug already fixed on `develop` by 
#1022, merged on 2026-10-18 as `963b55237f84224a958504dd31693dfcef0afb63`. It 
does not report a remaining bug on the current default branch. This issue 
should be marked completed using that existing fix; no additional fix is 
requested in #1026.
   
   ### Observed behavior
   
   C++ DICTIONARY decoding truncated a TEXT/STRING value at its first embedded 
NUL byte. For example, the three-byte string `std::string("a\0b", 3)` (hex `61 
00 62`) was read back as the one-byte string `"a"` (hex `61`). The equivalent 
PLAIN-encoded string preserved all three bytes.
   
   The behavior was reproduced while reviewing the standalone branch of #1026, 
whose head `c01ee105d397b1ed16a4daf335db9d8176036bac` does not contain the 
independently merged mainline fix. It was also reproduced on that PR's base 
commit `1180b13675e11a0230708af2cc70d037e0e12623`; this is not a regression 
introduced by #1026. No released-version range is claimed here.
   
   ### Reproduction
   
   1. Create a C++ table with a STRING field using DICTIONARY encoding and 
UNCOMPRESSED compression.
   2. Write the values `std::string("a\0b", 3)`, an empty string, and `"z"`, 
passing their explicit byte lengths to `common::String`.
   3. Close and reopen the file, read the table, and compare both the byte 
lengths and contents.
   4. On the affected code, the first value has length 1 instead of 3. 
Repeating the same round trip with PLAIN encoding preserves length 3.
   
   Expected: DICTIONARY preserves the complete byte sequence, including 
embedded and trailing NUL bytes.
   
   ### Root cause and existing resolution
   
   `DictionaryDecoder` uses `SerializationUtil::read_var_str(std::string&, 
ByteStream&)`. The affected implementation decoded the encoded length but 
constructed the output using `std::string(tmp_buf)`, which stops at the first 
NUL byte.
   
   #1022 changes this to `str.assign(tmp_buf, len)` and adds byte-stream and 
dictionary regression coverage, including a Java-generated dictionary fixture. 
The current default branch, `develop`, already contains this fix. This record 
documents the review finding and its existing resolution rather than requesting 
duplicate work.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to