ColinLeeo opened a new issue, #1080: URL: https://github.com/apache/tsfile/issues/1080
## Summary Applications that load training data issue one time-range query per window. On that access pattern the public table query API costs a **fixed ~0.30 ms per call**, almost independent of the number of rows returned: 64 windows (2048 points each) take 20.6–23.1 ms, while a single prepared-series query for the same window takes 0.046 ms. `prepare_series` + `query_prepared` already avoid most of the per-call cost, but the prepared path is (a) only reachable through locators produced by the Python dataset index, and (b) has several defects that limit how much it helps. This issue records what we measured and the concrete fixes that would make prepared series the normal fast path. ## Workload - 323 regular series × 105,120 points, 5-minute interval, one table, one string TAG (`series_id`). - Values: float32 **PLAIN**; time: **TS_2DIFF**; no compression; page size 10,000 points. - One window = 2048 rows; 64 windows per batch; warm page cache; one CPU; `threads=1`; medians over repeated runs. - AMD EPYC 9J14, Python 3.11.18, tsfile 2.5.0 wheel plus locally built `develop` (identical CMake flags). ## Measurements (ms per single-window call) | API path | ms/window | |---|---:| | `query_table`, `batch_size = window length` | 0.32–0.36 | | `query_prepared`, value column, time range, `limit = length` | **0.046** | | `query_prepared`, value column, time range, no `limit` | 0.079 | | `query_prepared`, `valid` column only | 0.076 | | two single-column `query_prepared` calls (value + valid) | 0.121 | | `query_prepared_multi`, two columns, time range | 0.26–0.27 | | `query_prepared_multi`, two columns, `offset/limit` | **1.79** | | prepared-handle acquire | 0.004 | Per 64-window batch: 20.6–23.1 ms with `query_table`, ~4–5 ms with prepared queries. The cost is not proportional to rows: a 0-row query still costs 0.325 ms, a 10,000-row query 0.55 ms, and a full 105,120-row query 0.92 ms (8.8 ns/row). ## Where the time goes per `query_table` call `query_table()` itself is lazy (~0.008 ms). The first `read_arrow_batch()` performs all of the work: 1. The table-level device index leaf node is read and deserialized to apply the tag filter. 2. `SingleDeviceTsBlockReader::init_internal()` calls `TsFileIOReader::get_timeseries_indexes()`, which re-reads and deserializes the device metadata node via `load_device_index_entry()` and does **not** consult `device_node_cache_`, even though `alloc_multi_ssi()` later uses the cached node. Measured: repeated queries on the same series (cache hot) cost 0.316 ms vs 0.440 ms when cycling 323 series — the cache only saves ~28%. 3. A new `TsFileSeriesScanIterator` + `ChunkReader` is constructed and destroyed per query. 4. `ChunkReader::get_next_page()` walks page headers from the start of the chunk, reading `DEFAULT_READ_SIZE = 4096` bytes per header; a 2048-point window triggers ~14 such reads (~57 KiB) even when the filter covers only the first page. There is no early stop once the page range is past the filter. 5. The intersecting page is decoded in full (10,000 points for a 2048-point window). 6. The `TsBlock` is allocated per query with `max_row_count = batch_size` (e.g. 65,536 → ~0.8 MB); this alone costs 0.079 vs 0.040 ms. The profile of the per-query work is dominated by allocator traffic and metadata construction; actual decoding (`multi_DECODE_TV_BATCH` + TS_2DIFF + time filter) is only ~6.5%. ## Prepared path defects - `PreparedSeriesTsBlockReader` hard-codes `PREPARED_BATCH_ROWS = 65536`, so without an explicit `limit` every prepared query preallocates a 65,536-row block. Passing `limit` halves the call cost (0.079 → 0.046 ms). - Every prepared call still constructs a new SSI/`ChunkReader`; `PreparedSeries` only caches the resolved index. - `query_prepared_multi(..., offset, limit)` degrades to 1.79 ms/call, ~40× a single-column prepared call and ~7× the two-column time-range variant. - Two single-column prepared calls (0.121 ms) are ~2.2× faster than one `query_prepared_multi` (0.27 ms) for two columns. - The prepared locator is only produced by the Python dataset index (`TsFileDataFrame(use_index=True)`), so users not using that layer cannot call `prepare_series` by (table, tag values, field). ## Proposed changes 1. Make `get_timeseries_indexes()` reuse the parsed device node from `device_node_cache_`, or drop the redundant call in `init_internal` since `alloc_multi_ssi()` already resolves the same node. 2. Stop scanning page headers once the filter range is exhausted, and/or use a page index to seek. 3. Push the row offset down to page granularity so a 2048-point window does not decode a 10,000-point page. 4. In the prepared path: honor the requested row count in `PREPARED_BATCH_ROWS`, reuse the SSI/`ChunkReader` across calls on the same prepared series, and expose a public `prepare(table, tags, field)` entry point. 5. Fix `query_prepared_multi` with `offset/limit` and investigate why two columns are slower than two single-column calls. ## Notes - PR #1072 (wide-table reads) does not change this workload: with the same build configuration it executes the same number of instructions as `develop` (8.824G vs 8.825G for 1500 queries), because this dataset uses PLAIN values (no RLE) and one measurement per device (no per-measurement index fan-out). - Separately, we measured current `develop` at ~8.8G instructions vs 7.2G for 2.5.0 on the same 1500-query loop (both locally built with identical CMake flags), i.e. ~21% more instructions per query. We have not bisected this yet; it may deserve its own issue. - The benchmark script and the dataset generator parameters can be shared if useful. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
