Hi all,

I would like to start a discussion about two related pieces of work on the
C++ reader, before anyone writes code for them. Neither is a small change,
and both touch interfaces that other code (and other language bindings)
depend on, so early feedback would be very useful.


------------------------------------------------------------------
Current state (as a shared baseline)
------------------------------------------------------------------


The table read path is organized around one device at a time:


  TableQueryExecutor
    -> DeviceOrderedTsBlockReader      (single-threaded, pulls one device)
         -> DeviceTaskIterator          (walks the device metadata index)
         -> SingleDeviceTsBlockReader   (per device: builds one TsBlock)
              -> TsFileSeriesScanIterator -> ChunkReader / AlignedChunkReader


The only concurrency in the read path today is aligned multi-column page
decode, submitted to the process-wide thread pool and gated on
(num_columns > 1 && parallel_read_enabled_). A narrow query over many
devices runs entirely on one thread.


The in-memory block (common::TsBlock) has three properties that matter
for what follows:


  1. The iteration cursor lives inside the block: RowIterator/ColIterator
     advance a mutable offset stored in each Vector. Two consumers cannot
     read the same block concurrently. This is also why a scan reuses a
     single output block and requires the consumer to finish with it
     before the next call.
  2. Nulls are dense: a null row occupies no slot in the value buffer,
     only a bit in the validity bitmap.
  3. Variable-length values are stored as a 4-byte length prefix
     followed by the bytes, inline in the value buffer.


On export, cwrapper converts each block to the Arrow C Data Interface
(hand-written; we deliberately do not link Arrow C++ so the build stays
C++11). That conversion allocates new buffers and copies every column.
TsBlock itself never crosses the C ABI: Python and Go both use
tsfile_result_set_get_next_tsblock_as_arrow().


------------------------------------------------------------------
Direction 1: make the internal read block Arrow-native (C++)
------------------------------------------------------------------


Goal: one block layout, from page decode to the language binding, with
no conversion on the way out.


  - Introduce a small common::Block that owns an ArrowArray/ArrowSchema
    pair (same layout the existing exporter already produces), held by
    shared_ptr, with typed column views. No Arrow C++ dependency; C++11
    is preserved.
  - TsFileSeriesScanIterator produces Block; the table readers produce
    Block; cwrapper hands the buffers over by transferring ownership
    (zero copy) instead of converting.
  - TsBlock is kept as an assembly shim only where it is still needed:
    the tree-view QDS readers and some unit tests. It becomes an
    internal compatibility path, not the primary representation.
  - Java is explicitly out of scope for the first phase. Its TsBlock /
    Column classes are a public read API, so it needs a separate
    coexistence/deprecation plan.


Why this ordering matters: immutable, reference-counted blocks are also
the prerequisite for Direction 2. A parallel producer pipeline needs to
hand blocks to a consumer while continuing to work, which the current
mutable-cursor, single-reused-block design cannot support safely.


Known costs and risks:


  - Dense nulls become slot-aligned. Fixed-width columns will keep a
    placeholder slot for null rows, so a sparse column can take more
    memory than today. We plan to measure the null-slot ratio on
    representative data before committing to this part.
  - A block exported to pyarrow is owned by the consumer and may be held
    indefinitely, so block recycling must be reference-count based.
  - Row-oriented accessors (ResultSet::next(), get_*_at()) must read the
    Arrow buffers directly. Assembling a TsBlock there would add a full
    block copy to the Python row path.


------------------------------------------------------------------
Direction 2: scan at cross-device granularity
------------------------------------------------------------------


Goal: stop treating "one device" as the unit of work.


  - A task becomes a (file, device range, time range) unit, sized by
    estimated bytes rather than by device boundaries.
  - A bounded pool of workers produces blocks into a merge buffer; the
    existing pull interface (TsBlockReader::has_next/next, ResultSet)
    stays unchanged, so no consumer has to be rewritten.
  - Device ordering is preserved by sequence number. Time ordering is
    currently unsupported (the C++ executor returns E_UNSUPPORTED_ORDER);
    with independent per-device streams it becomes a lazy k-way merge
    over block streams rather than a physical row-by-row merge.
  - OFFSET/LIMIT need cancellation semantics: an atomic budget plus
    cancellation of not-yet-started tasks, replacing the current
    sequential remaining_offset_/remaining_limit_ accounting.


Prerequisites we have identified:


  - TsFileIOReader lazy-loads file metadata behind a plain bool; this
    needs a once-guard or eager initialization before workers start.
    device_node_cache_ is already mutex-protected, and every
    RandomAccessReadFile implementation documents concurrent read() as
    safe (pread based), so the I/O side is fine.
  - Per-worker arenas; the allocator's statistics are atomic.
  - Nested parallelism: AlignedChunkReader submits column tasks to the
    shared pool and waits on futures. If device-level workers are
    themselves pool threads that wait on column tasks, the pool can
    deadlock. Our first approach is to pick one axis per query (device
    parallelism for narrow projections, column parallelism for wide
    ones) rather than nesting them.


------------------------------------------------------------------
Proposed phasing
------------------------------------------------------------------


  P0  common::Block + block pool + TsBlock assembler + tests.
      No behavior change; independently testable.
  P1  TsFileSeriesScanIterator and the chunk readers produce Block,
      including the existing direct-decode path.
  P2  Table readers, ResultSet row accessors, zero-copy C ABI export.
  P3  Cross-device scheduler and ordered merge on top of P0-P2.
  P4  Tree-view migration, or keep the assembler long term.


C++ first. Python and Go need no API change: they keep using the same
C functions, which simply stop copying.


For context, on our internal wide-table benchmark (about 4,500 devices,
monthly files), a narrow 5-column, one-month scan takes roughly 1.4 s on
a single thread after all the index/decode work already merged. What is
left is serialized index I/O and single-threaded decode, which is why
Direction 2 is the one that can still scale with hardware.


------------------------------------------------------------------
Questions for the community
------------------------------------------------------------------


  1. Does anyone depend on the current TsBlock C++ API directly, outside
     this repository? That determines whether the assembly shim has to be
     a stable interface or can be internal-only.
  2. Is TIME-ordered output needed by downstream consumers, or is device
     order sufficient? Supporting it changes the merge design.
  3. Is the memory cost of slot-aligned nulls acceptable for sparse
     measurements, or should we keep dense storage internally and align
     only at the export boundary?
  4. Should cross-device parallelism be opt-in via configuration (like
     parallel_read_enabled_) initially, or on by default?
  5. For Java, is there interest in a parallel scan design now, or should
     that be a separate discussion later?


Nothing here is committed yet; I would like to agree on scope and
ordering first, and I can follow up with a detailed design document and
an umbrella GitHub issue once we have direction.


Thanks,
Colin

Reply via email to