Hi everyone,
I would like to discuss creating a maintained performance and resource reference for Apache TsFile, together with a companion command-line tool tentatively named `tsfile-benchmark`. The community has already accumulated benchmark programs, test results, and performance investigations. However, we do not yet have one public, repeatable reference that users can consult when they ask practical questions such as: - How much memory does TsFile use for this workload? - Does continuous writing remain stable over a long period? - What changes when the application runs with limited memory? - What read performance can I expect on my own edge device? Today, these questions often lead to an ad hoc measurement for an individual user. Those results can be useful, but they are difficult for other users to find, reproduce, or compare across versions and devices. Similar questions therefore require similar measurements again and again. My proposal is to consolidate and extend the work we already have, rather than replace it: 1. **A Performance and Resource Reference.** A versioned, community-maintained document published through the TsFile documentation after community review. It would describe measured behavior under clearly stated conditions, with the workloads, environments, configurations, raw results, and reproduction scripts available to readers. 2. **`tsfile-benchmark`.** An installable command-line tool that lets users run standard TsFile benchmarks on their own machines or edge devices and save the result locally in a readable and machine-readable format. The two efforts are complementary. The reference would provide curated results from controlled environments; `tsfile-benchmark` would let a user characterize the device and files they actually use. Where practical, they should share workload definitions and result formats, but a local result should be interpreted in the context of its own device and configuration rather than as a universal score. For the reference, I suggest that the initial focus be the questions users currently find hardest to answer: - peak and steady-state memory use during writes; - throughput and memory stability during long-running writes; - behavior when the application is subject to practical memory constraints; - the trade-offs among write throughput, flush behavior, and resulting file size; and - read throughput and latency on a small set of reference devices. For `tsfile-benchmark`, the initial scope should include both read and write benchmarks, together with the resource measurements needed to interpret them. Read workloads can include scans, time-range queries, point or small-range lookups, and representative multi-series queries. Write workloads can cover sustained writes, bounded-memory configurations, flush behavior, throughput, memory use, and resulting file size. The tool should record enough context to make a local run meaningful, including TsFile and tool versions, workload parameters, file or dataset identity, operating system, CPU architecture, memory, storage, and relevant configuration. A user should be able to keep the resulting JSON or text summary with a support question or use it to compare versions on the same device. To keep the first iteration focused, I suggest we start from the C++ core and existing benchmark work, rather than try to cover every binding and platform at once. Java and Python behavior, additional architectures, and further workloads can follow once we agree on a useful initial baseline. Before proposing a detailed design, I would like to collect feedback on four questions: 1. Which deployment scenarios should the first reference and tool prioritize, especially for edge deployments? 2. Which questions and metrics matter most to users—for example, peak memory, memory after close, long-running write stability, read throughput or latency, flush behavior, and file size? 3. Which device classes, storage media, memory conditions, workloads, or datasets would be representative enough to include first? 4. Are community members willing to share existing benchmark programs, representative data, reference hardware, or methodology review? Please reply in this thread with your feedback. After collecting the discussion, I will summarize the feedback and propose a small initial scope, metric definitions, and a reproducible methodology for a follow-up discussion. Any project-maintained tool or published reference would proceed through the normal community review process. Thank you for your feedback. Best regards, Colin
