ColinLeeo opened a new issue, #1034:
URL: https://github.com/apache/tsfile/issues/1034
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
This is a historical tracking issue for a bug already fixed on `develop` by
#1022, merged on 2026-10-18 as `963b55237f84224a958504dd31693dfcef0afb63`. It
does not report a remaining bug on the current default branch. This issue
should be marked completed using that existing fix; no additional fix is
requested in #1026.
### Observed behavior
C++ DICTIONARY decoding truncated a TEXT/STRING value at its first embedded
NUL byte. For example, the three-byte string `std::string("a\0b", 3)` (hex `61
00 62`) was read back as the one-byte string `"a"` (hex `61`). The equivalent
PLAIN-encoded string preserved all three bytes.
The behavior was reproduced while reviewing the standalone branch of #1026,
whose head `c01ee105d397b1ed16a4daf335db9d8176036bac` does not contain the
independently merged mainline fix. It was also reproduced on that PR's base
commit `1180b13675e11a0230708af2cc70d037e0e12623`; this is not a regression
introduced by #1026. No released-version range is claimed here.
### Reproduction
1. Create a C++ table with a STRING field using DICTIONARY encoding and
UNCOMPRESSED compression.
2. Write the values `std::string("a\0b", 3)`, an empty string, and `"z"`,
passing their explicit byte lengths to `common::String`.
3. Close and reopen the file, read the table, and compare both the byte
lengths and contents.
4. On the affected code, the first value has length 1 instead of 3.
Repeating the same round trip with PLAIN encoding preserves length 3.
Expected: DICTIONARY preserves the complete byte sequence, including
embedded and trailing NUL bytes.
### Root cause and existing resolution
`DictionaryDecoder` uses `SerializationUtil::read_var_str(std::string&,
ByteStream&)`. The affected implementation decoded the encoded length but
constructed the output using `std::string(tmp_buf)`, which stops at the first
NUL byte.
#1022 changes this to `str.assign(tmp_buf, len)` and adds byte-stream and
dictionary regression coverage, including a Java-generated dictionary fixture.
The current default branch, `develop`, already contains this fix. This record
documents the review finding and its existing resolution rather than requesting
duplicate work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]