Hi all,
I would like to start a discussion about two related pieces of work on the C++ reader, before anyone writes code for them. Neither is a small change, and both touch interfaces that other code (and other language bindings) depend on, so early feedback would be very useful. ------------------------------------------------------------------ Current state (as a shared baseline) ------------------------------------------------------------------ The table read path is organized around one device at a time: TableQueryExecutor -> DeviceOrderedTsBlockReader (single-threaded, pulls one device) -> DeviceTaskIterator (walks the device metadata index) -> SingleDeviceTsBlockReader (per device: builds one TsBlock) -> TsFileSeriesScanIterator -> ChunkReader / AlignedChunkReader The only concurrency in the read path today is aligned multi-column page decode, submitted to the process-wide thread pool and gated on (num_columns > 1 && parallel_read_enabled_). A narrow query over many devices runs entirely on one thread. The in-memory block (common::TsBlock) has three properties that matter for what follows: 1. The iteration cursor lives inside the block: RowIterator/ColIterator advance a mutable offset stored in each Vector. Two consumers cannot read the same block concurrently. This is also why a scan reuses a single output block and requires the consumer to finish with it before the next call. 2. Nulls are dense: a null row occupies no slot in the value buffer, only a bit in the validity bitmap. 3. Variable-length values are stored as a 4-byte length prefix followed by the bytes, inline in the value buffer. On export, cwrapper converts each block to the Arrow C Data Interface (hand-written; we deliberately do not link Arrow C++ so the build stays C++11). That conversion allocates new buffers and copies every column. TsBlock itself never crosses the C ABI: Python and Go both use tsfile_result_set_get_next_tsblock_as_arrow(). ------------------------------------------------------------------ Direction 1: make the internal read block Arrow-native (C++) ------------------------------------------------------------------ Goal: one block layout, from page decode to the language binding, with no conversion on the way out. - Introduce a small common::Block that owns an ArrowArray/ArrowSchema pair (same layout the existing exporter already produces), held by shared_ptr, with typed column views. No Arrow C++ dependency; C++11 is preserved. - TsFileSeriesScanIterator produces Block; the table readers produce Block; cwrapper hands the buffers over by transferring ownership (zero copy) instead of converting. - TsBlock is kept as an assembly shim only where it is still needed: the tree-view QDS readers and some unit tests. It becomes an internal compatibility path, not the primary representation. - Java is explicitly out of scope for the first phase. Its TsBlock / Column classes are a public read API, so it needs a separate coexistence/deprecation plan. Why this ordering matters: immutable, reference-counted blocks are also the prerequisite for Direction 2. A parallel producer pipeline needs to hand blocks to a consumer while continuing to work, which the current mutable-cursor, single-reused-block design cannot support safely. Known costs and risks: - Dense nulls become slot-aligned. Fixed-width columns will keep a placeholder slot for null rows, so a sparse column can take more memory than today. We plan to measure the null-slot ratio on representative data before committing to this part. - A block exported to pyarrow is owned by the consumer and may be held indefinitely, so block recycling must be reference-count based. - Row-oriented accessors (ResultSet::next(), get_*_at()) must read the Arrow buffers directly. Assembling a TsBlock there would add a full block copy to the Python row path. ------------------------------------------------------------------ Direction 2: scan at cross-device granularity ------------------------------------------------------------------ Goal: stop treating "one device" as the unit of work. - A task becomes a (file, device range, time range) unit, sized by estimated bytes rather than by device boundaries. - A bounded pool of workers produces blocks into a merge buffer; the existing pull interface (TsBlockReader::has_next/next, ResultSet) stays unchanged, so no consumer has to be rewritten. - Device ordering is preserved by sequence number. Time ordering is currently unsupported (the C++ executor returns E_UNSUPPORTED_ORDER); with independent per-device streams it becomes a lazy k-way merge over block streams rather than a physical row-by-row merge. - OFFSET/LIMIT need cancellation semantics: an atomic budget plus cancellation of not-yet-started tasks, replacing the current sequential remaining_offset_/remaining_limit_ accounting. Prerequisites we have identified: - TsFileIOReader lazy-loads file metadata behind a plain bool; this needs a once-guard or eager initialization before workers start. device_node_cache_ is already mutex-protected, and every RandomAccessReadFile implementation documents concurrent read() as safe (pread based), so the I/O side is fine. - Per-worker arenas; the allocator's statistics are atomic. - Nested parallelism: AlignedChunkReader submits column tasks to the shared pool and waits on futures. If device-level workers are themselves pool threads that wait on column tasks, the pool can deadlock. Our first approach is to pick one axis per query (device parallelism for narrow projections, column parallelism for wide ones) rather than nesting them. ------------------------------------------------------------------ Proposed phasing ------------------------------------------------------------------ P0 common::Block + block pool + TsBlock assembler + tests. No behavior change; independently testable. P1 TsFileSeriesScanIterator and the chunk readers produce Block, including the existing direct-decode path. P2 Table readers, ResultSet row accessors, zero-copy C ABI export. P3 Cross-device scheduler and ordered merge on top of P0-P2. P4 Tree-view migration, or keep the assembler long term. C++ first. Python and Go need no API change: they keep using the same C functions, which simply stop copying. For context, on our internal wide-table benchmark (about 4,500 devices, monthly files), a narrow 5-column, one-month scan takes roughly 1.4 s on a single thread after all the index/decode work already merged. What is left is serialized index I/O and single-threaded decode, which is why Direction 2 is the one that can still scale with hardware. ------------------------------------------------------------------ Questions for the community ------------------------------------------------------------------ 1. Does anyone depend on the current TsBlock C++ API directly, outside this repository? That determines whether the assembly shim has to be a stable interface or can be internal-only. 2. Is TIME-ordered output needed by downstream consumers, or is device order sufficient? Supporting it changes the merge design. 3. Is the memory cost of slot-aligned nulls acceptable for sparse measurements, or should we keep dense storage internally and align only at the export boundary? 4. Should cross-device parallelism be opt-in via configuration (like parallel_read_enabled_) initially, or on by default? 5. For Java, is there interest in a parallel scan design now, or should that be a separate discussion later? Nothing here is committed yet; I would like to agree on scope and ordering first, and I can follow up with a detailed design document and an umbrella GitHub issue once we have direction. Thanks, Colin
