Hi,  Hongyin,

Thank you for starting this discussion. I strongly welcome and support the 
introduction of `tsfile-cli`, which can become a valuable file-level tool in 
the TsFile ecosystem.

The value of a CLI has already been demonstrated in data format ecosystems such 
as Apache Parquet. For example, Apache Parquet's `parquet-cli` allows users to 
inspect the schema, metadata, and contents of a Parquet file directly, without 
first setting up a service or writing a program. `tsfile-cli` can provide a 
similar standard entry point for TsFile, significantly reducing the cost of 
exploring unfamiliar files, diagnosing problems, and performing batch 
processing. It can also make TsFile operations easy to integrate into shell 
pipelines and automated workflows.

A CLI is equally valuable for automation and AI agent scenarios. For many data 
inspection and processing tasks, writing a script against a stable command 
contract is usually faster than developing a complete application, while also 
being easier to reuse and integrate. Machine-readable output, well-defined exit 
codes, and consistent error handling can reduce the need for users or agents to 
guess the underlying APIs, file structure, or data semantics. This lowers 
implementation complexity and the risk of errors, while accelerating subsequent 
data analysis and processing workflows.

Regarding the boundary between the tools, I see `tsfile-cli` as a local file 
reader and writer with no server dependency, intended for terminal users and 
automation scripts. The TsFile SDKs provide programming interfaces for 
application development, while IoTDB tools focus on server-side data querying 
and management. Although using the CLI is not tied to a particular programming 
language, it is still a complete TsFile reader and writer at a level similar to 
the SDKs. I therefore believe it is appropriate to maintain it in the main 
TsFile repository. The current CLI is built directly on the TsFile C++ 
implementation and needs to remain closely compatible with its file format and 
read/write behavior. Maintaining, testing, versioning, and releasing them 
together can provide stronger compatibility guarantees and enable the community 
to maintain the CLI responsibly as an official, long-lived tool. Future work 
should prioritize functional completeness and consistency in command behavior, 
machine-readable output, error handling, and safety guarantees. The CLI can be 
released and installed together with TsFile so that both developers and end 
users can use it directly to improve the efficiency of data exploration and 
processing.

Overall, I strongly support continuing the development of `tsfile-cli`, and I 
welcome further community discussion about its scope, interface design, 
maintenance model, and release plan.


Best regards,
Colin Lee.





At 2026-08-17 17:33:51, "张洪胤" <[email protected]> wrote:
>Hi everyone,
>
>I would like to start a discussion about introducing tsfile-cli, a 
>file-oriented command-line interface for Apache TsFile.
>
>1. Motivation
>
>Many users first encounter a single .tsfile and need to answer relatively 
>simple questions: What storage model does it use? Which tables or devices does 
>it contain? What is the schema, time range, or row count? Can I inspect a few 
>rows, stream data through a pipe, or convert CSV/TSV data into a new TsFile?
>
>Today, answering these questions often requires writing a C++/Java/Python 
>program, constructing a reader, or using several specialized tools. The 
>existing tools remain valuable, but they address different use cases: 
>low-level physical debugging, bulk data import, SDK/API access, or queries 
>against an IoTDB server.
>
>What is currently missing is a single, local, and pipe-friendly entry point 
>for working with an individual TsFile.
>
>2. AI-Assisted Access
>
>This capability is also increasingly important for AI-assisted workflows. An 
>AI assistant cannot safely infer the schema and semantics of a binary TsFile.
>
>Without a stable command-line contract, it may generate one-off programs, 
>guess at APIs, or repeatedly retry after ambiguous failures. A predictable CLI 
>with machine-readable output, stable exit codes, explicit error handling, and 
>a maintained SKILL.md gives both users and AI assistants a reliable way to 
>inspect and process TsFile files.
>
>3. Scope and Boundaries
>
>tsfile-cli is intended to complement the existing SDKs and IoTDB tools, not 
>replace them.
>
>It does not introduce a new storage engine or a server-side query layer. 
>Instead, it provides a thin, file-level interface on top of the existing 
>TsFile C++ APIs.
>
>The initial scope focuses on one local TsFile at a time, covering common 
>inspection, query, conversion, and automation tasks without requiring an IoTDB 
>server or a language-specific application.
>
>4. Current Implementation
>
>The initial implementation is already available in the repository under 
>cpp/tools/.
>
>It currently includes commands for:
>
>listing tables or devices;
>
>inspecting schema and file metadata;
>
>collecting statistics and row counts;
>
>previewing, streaming, or sampling rows;
>
>importing CSV/TSV data into a new table-model TsFile.
>
>The implementation is built together with the C++ module. Its command behavior 
>and examples are documented in:
>
>cpp/tools/README.md
>
>cpp/tools/skills/tsfile-cli/SKILL.md
>
>The initial implementation was introduced in PR #829.
>
>5. Requirements Document
>
>The requirements document, including the proposed scope, command contracts, AI 
>integration, testing, and future capabilities, is available here:
>
>https://apache-iotdb-project.feishu.cn/docx/Hbd3dA7lco78ofxAn5jc3talnkb
>
>It describes the motivation, target users, tree/table model boundaries, 
>command contracts, output formats, error handling, safety requirements, AI 
>integration, testing, and possible future capabilities such as controlled file 
>export.
>
>The requirements document represents the proposed product direction. The 
>current implementation should be understood as an initial baseline, rather 
>than a claim that every proposed capability has already been completed.
>
>6. Contributions
>
>I would particularly like to thank Zeyu Zhang, who has contributed 
>substantially to this work.
>
>His contributions include multiple bug fixes and the addition of table-model 
>TAG filtering in PR #837. This work helped move tsfile-cli from an initial 
>prototype toward a more usable implementation.
>
>7. Feedback Requested
>
>I would appreciate feedback from the community on:
>
>whether a standard file-level CLI would be useful as part of the TsFile 
>distribution;
>
>the appropriate boundary between tsfile-cli, the SDKs, and IoTDB tools;
>
>the proposed command, output, and error contracts;
>
>the priority of the next implementation steps;
>
>how the CLI should eventually be packaged and released.
>
>Feedback on both the requirements document and the existing implementation 
>would be very welcome.
>
>Best regards,
>
>Zhang Hongyin

Reply via email to