ColinLeeo opened a new issue, #1140:
URL: https://github.com/apache/tsfile/issues/1140

   ### Background
   
   The C++ table read path is organized around one device at a time:
   
   ```
   TableQueryExecutor
     -> DeviceOrderedTsBlockReader      (single-threaded, pulls one device)
          -> DeviceTaskIterator          (walks the device metadata index)
          -> SingleDeviceTsBlockReader   (per device: builds one TsBlock)
               -> TsFileSeriesScanIterator -> ChunkReader / AlignedChunkReader
   ```
   
   The only concurrency in the read path today is aligned multi-column page 
decode, submitted to the process-wide thread pool and gated on `num_cols > 1 && 
parallel_read_enabled_` 
([cpp/src/reader/tsfile_series_scan_iterator.cc:473](https://github.com/apache/tsfile/blob/develop/cpp/src/reader/tsfile_series_scan_iterator.cc#L473)).
 A narrow query over many devices runs entirely on one thread: index I/O and 
decode are serialized per device.
   
   On our internal wide-table benchmark (about 4,500 devices, monthly files), a 
narrow 5-column, one-month scan takes roughly 1.4 s on a single thread after 
the index/decode work already merged. What is left is serialized index I/O and 
single-threaded decode, which is what this issue targets.
   
   Additional constraints verified in the code:
   
   - `DeviceOrderedTsBlockReader` carries 
`remaining_offset_`/`remaining_limit_` and decrements them sequentially as 
devices complete 
([cpp/src/reader/block/device_ordered_tsblock_reader.h:43](https://github.com/apache/tsfile/blob/develop/cpp/src/reader/block/device_ordered_tsblock_reader.h#L43)).
   - TIME-ordered output is currently rejected with `E_UNSUPPORTED_ORDER` 
([cpp/src/reader/table_query_executor.cc:103](https://github.com/apache/tsfile/blob/develop/cpp/src/reader/table_query_executor.cc#L103)).
   - The current block is mutable and cursor-bearing, so a block cannot safely 
be handed from a producer thread to the consumer while decode continues; that 
is why this work depends on the Arrow-native block issue.
   
   ### Proposal
   
   Stop treating "one device" as the unit of work.
   
   - A task becomes a `(file, device range, time range)` unit, sized by 
estimated bytes rather than by device boundaries.
   - A bounded pool of workers produces blocks into a merge buffer; the 
existing pull interface (`TsBlockReader::has_next/next`, `ResultSet`) stays 
unchanged, so no consumer has to be rewritten.
   - Device ordering is preserved by sequence number. TIME ordering becomes a 
lazy k-way merge over per-device block streams rather than a physical 
row-by-row merge.
   - OFFSET/LIMIT need cancellation semantics: an atomic budget plus 
cancellation of not-yet-started tasks, replacing the current sequential 
`remaining_offset_`/`remaining_limit_` accounting.
   - Cross-device parallelism should be opt-in via configuration (like 
`parallel_read_enabled_`) initially.
   
   This also addresses the latency-overlap problem for remote storage (#1138): 
prefetching the next chunk(s) on the worker pool hides object-storage round 
trips behind decode.
   
   ### Dependencies
   
   Blocked on the Arrow-native block work (P0-P2): immutable, reference-counted 
blocks are required to hand data between producer and consumer safely. The 
scheduler (P3 in the mailing-list phasing) can start designing in parallel but 
cannot land first.
   
   ### Acceptance criteria
   
   - Multi-device scan scales with cores: e.g. >= 2x throughput on a 4-core 
machine for the wide-table benchmark above, reported with thread count and 
hardware.
   - Device-order output is bit-identical to the sequential path; OFFSET/LIMIT 
results identical for all combinations of offset/limit, including `limit == 0` 
and limits that cut a device mid-block.
   - LIMIT short-circuits: a query with `LIMIT 10` over 4,500 devices must not 
scan every device.
   - Thread-safety under TSan; no regression for single-device queries or when 
the feature is disabled.
   - Local-file and remote (seekable) sources both exercise the new path.
   
   ### Open questions
   
   1. Is TIME-ordered output needed by downstream consumers, or is device order 
sufficient? Supporting it changes the merge design.
   2. Should cross-device parallelism be opt-in initially, or on by default 
once stable?
   3. For Java, is there interest in a parallel scan design now, or should that 
be a separate discussion later?
   
   Related: Arrow-native block issue (dependency); #1138 object-storage read 
latency; #1080 point-query metadata rebuild; #1114/#1115 SPRINTZ decode cost.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to