Hi all,
Apache TsFile 2.5.0 was released on October 14, and develop is now on
2.5.1-SNAPSHOT. I have created the 2.5.1 milestone to track the scope of the
next release:
https://github.com/apache/tsfile/milestone/1
Below is the current proposed scope and the reason behind each item. This is a
starting point rather than a fixed list, and feedback is very welcome.
1. Already fixed
#1038 [Bug][C++] TimeIn query drops rows after a fully filtered aligned page
Fixed by #1044 (merged). A TimeIn query could silently omit matching rows
once an aligned page produced no matching timestamps; the scanner now keeps
reading the remaining pages of the chunk. Included in 2.5.1 because it is a
silent data-loss bug.
Bug fixes already merged into develop since 2.5.0 (for example #1019, #1022,
#1026 and #1045) will naturally be part of 2.5.1 as well. Add the corresponding
issues to the milestone if you want them tracked in the release notes.
2. Storage backends
#1054 Support and verify reading TsFiles on NFS-mounted paths
TsFile is increasingly used on NAS and on-premises storage. Nothing in the
repository currently documents or tests NFS behavior. The goal is to verify
the read path (default pread and opt-in mmap backends), multi-client
append/replace scenarios, and the size/mtime fingerprint used by the dataset
index, then fix what does not work or document the recommended configuration
and limitations.
#1055 [C++] Object storage (OSS) read and write support
Reading has already been solved for Python through user-provided seekable
file objects (#930), and the C++ reader accepts a pluggable
RandomAccessReadFile source. Writing has no equivalent abstraction in
C++/C/Python, and Java's os:// path depends on classes outside this
repository. The goal is to introduce a write-sink interface mirroring
RandomAccessReadFile, ship an in-memory implementation, and support writing
to object storage directly or through a spool-then-upload design, with tests
for the Java path as well.
3. Data model and format
#1058 Support adding, dropping, and renaming columns (schema evolution)
Long-lived tables need to evolve. Today the file metadata stores one
TableSchema per table, C++ rejects re-registering a table, and Java rejects
unknown columns and type changes. The goal is to define writer APIs and the
metadata/read semantics for adding a column (nulls in older chunk groups),
dropping a column (tombstone), and renaming a column (alias mapping), with
cross-language compatibility.
#1056 Allow non-decreasing (duplicate) timestamps behind an opt-in flag
Both implementations currently reject time == last_time, which forces
upstream deduplication. The goal is an opt-in writer setting that accepts
non-decreasing timestamps while still rejecting decreasing ones, together
with an audit of the encoders, readers, and duplicate semantics.
4. Compatibility and testing
#1057 Expand the Java/C++ cross-language compatibility matrix to full coverage
The current compatibility CI covers the table model only and a subset of
data types and encodings. The goal is a single shared matrix covering every
supported (data type, encoding, compression) pair, both models, and
boundary/scale cases, validated in both writing directions.
Call for proposals
2.5.1 is not limited to the list above. If there is a bug fix or feature you
think should ship in this release, please reply to this thread or comment on
the relevant issue/PR and add it to the milestone. It would help to have
proposals by November 9 so we can finalize the scope.
Since several items above are non-trivial, I would also welcome opinions on
whether any of them should be split or deferred, so that 2.5.1 stays focused.
Thanks,
Colin
At 2026-11-02 16:09:37, "ColinLee" <[email protected]> wrote:
>Hi all,Apache TsFile 2.5.0 was released on October 14, and develop is now
>on2.5.1-SNAPSHOT. I have created the 2.5.1 milestone to track the scope of
>thenext release: https://github.com/apache/tsfile/milestone/1Below is the
>current proposed scope and the reason behind each item. This is astarting
>point rather than a fixed list, and feedback is very welcome.1. Already
>fixed#1038 [Bug][C++] TimeIn query drops rows after a fully filtered aligned
>page Fixed by #1044 (merged). A TimeIn query could silently omit matching
>rows once an aligned page produced no matching timestamps; the scanner now
>keeps reading the remaining pages of the chunk. Included in 2.5.1 because it
>is a silent data-loss bug.Bug fixes already merged into develop since 2.5.0
>(for example #1019, #1022,#1026 and #1045) will naturally be part of 2.5.1 as
>well. Add the correspondingissues to the milestone if you want them tracked in
>the release notes.2. Storage backends#1054 Support and verify reading TsFiles
>on NFS-mounted paths TsFile is increasingly used on NAS and on-premises
>storage. Nothing in the repository currently documents or tests NFS behavior.
>The goal is to verify the read path (default pread and opt-in mmap backends),
>multi-client append/replace scenarios, and the size/mtime fingerprint used by
>the dataset index, then fix what does not work or document the recommended
>configuration and limitations.#1055 [C++] Object storage (OSS) read and write
>support Reading has already been solved for Python through user-provided
>seekable file objects (#930), and the C++ reader accepts a pluggable
>RandomAccessReadFile source. Writing has no equivalent abstraction in
>C++/C/Python, and Java's os:// path depends on classes outside this
>repository. The goal is to introduce a write-sink interface mirroring
>RandomAccessReadFile, ship an in-memory implementation, and support writing
>to object storage directly or through a spool-then-upload design, with tests
>for the Java path as well.3. Data model and format#1058 Support adding,
>dropping, and renaming columns (schema evolution) Long-lived tables need to
>evolve. Today the file metadata stores one TableSchema per table, C++ rejects
>re-registering a table, and Java rejects unknown columns and type changes.
>The goal is to define writer APIs and the metadata/read semantics for adding
>a column (nulls in older chunk groups), dropping a column (tombstone), and
>renaming a column (alias mapping), with cross-language compatibility.#1056
>Allow non-decreasing (duplicate) timestamps behind an opt-in flag Both
>implementations currently reject time == last_time, which forces upstream
>deduplication. The goal is an opt-in writer setting that accepts
>non-decreasing timestamps while still rejecting decreasing ones, together
>with an audit of the encoders, readers, and duplicate semantics.4.
>Compatibility and testing#1057 Expand the Java/C++ cross-language
>compatibility matrix to full coverage The current compatibility CI covers the
>table model only and a subset of data types and encodings. The goal is a
>single shared matrix covering every supported (data type, encoding,
>compression) pair, both models, and boundary/scale cases, validated in both
>writing directions.Call for proposals2.5.1 is not limited to the list above.
>If there is a bug fix or feature youthink should ship in this release, please
>reply to this thread or comment onthe relevant issue/PR and add it to the
>milestone. It would help to haveproposals by November 9 so we can finalize the
>scope.Since several items above are non-trivial, I would also welcome opinions
>onwhether any of them should be split or deferred, so that 2.5.1 stays
>focused.Thanks,Colin