hongzhi-gao opened a new pull request, #18915:
URL: https://github.com/apache/iotdb/pull/18915

   ## Description
   
   Reduce first-query aggregation latency introduced by #18606 by moving 
exhaustive type-coverage validation out of `TypeServices.Aggregation` class 
initialization.
   
   ### Problem and fix
   
   A simple `SELECT LAST ...` initializes `TypeServices.Aggregation`. Its 
static initializer calls `.check()` on six services, traversing all data types 
and constructing strategies unrelated to the requested aggregation. In 
particular, checking the max/min-by strategy also initializes the calc 
type-service registry. `max_time` queries can hit the same initialization path 
through aggregation size calculation.
   
   This work is paid on first use. A benchmark that restarts the database 
before each query workload repeatedly includes that cost in its reported 
average. In a separate pre/post-#18606 diagnostic, the first LAST query 
increased from 315 ms to 433 ms, while the other 99 queries averaged 9.95 ms 
and 9.86 ms respectively. That adjacent-commit comparison also changed the 
TsFile dependency; the controlled fix comparison below keeps the source 
revision and dependencies constant.
   
   Move the six existing checks, unchanged and in the same order, into 
`TypeServicesTest.testAggregationServicesCoverAllTypes()`. The checks construct 
and discard strategies; they do not perform required registration. Actual 
strategy factories, supported-type dispatch, aggregation algorithms and 
exception behavior at use sites remain unchanged. Real max/min-by queries still 
initialize the calc services they need. There are no file-format, 
configuration, persistence or concurrency-mechanism changes.
   
   ### Performance results
   
   The controlled server comparison uses unmodified 
`a670bce7ec6b9e16ebdd8c9beb09197058785865` (#18606) versus the same revision 
plus this patch, with the same TsFile `2.4.1-260915-SNAPSHOT` and benchmark 
client libraries. These numbers were measured on that historical revision, not 
on the latest master base of this PR. The complete unpatched `Aggregation` 
class is identical on that revision and the PR base (`4c34a253927`).
   
   Environment: 8 vCPUs, 31 GiB RAM, Java 17.0.9, VMware, ext4; one ConfigNode 
and one DataNode, 20 GiB DataNode heap, all three compaction modes disabled. 
The existing aligned dataset contains 2,402 TsFiles (approximately 16 GiB). 
Each round restores the copied cluster metadata and starts fresh database JVMs. 
A private network namespace avoids collisions with another workload on the same 
host. Host page caches were not explicitly evicted in this matrix.
   
   Single benchmark client; Q8 performs 100 LAST queries per round over a fixed 
workload with 100 devices and 300 measurements (query seed 151658); Q4b-3 
performs 20 time-range `max_time` queries per round. Three 
alternating/reversed-order pairs per workload, without the per-query timing 
agent:
   
   | Workload | Unpatched round means (ms) | Patched round means (ms) | Mean 
before → after | Reduction |
   | --- | --- | --- | --- | --- |
   | Q8 (`LAST`) | 14.24 / 10.62 / 13.96 | 12.96 / 9.77 / 10.24 | 12.94 → 10.99 
ms | 15.1% |
   | Q4b-3 (`max_time`) | 52.02 / 44.07 / 32.85 | 27.57 / 26.99 / 27.90 | 42.98 
→ 27.49 ms | 36.0% |
   
   An additional, separately instrumented Q8 pair measured first-query latency 
of **388.27 → 308.97 ms**. The following 99 queries averaged **10.67 → 11.25 
ms**, so this is evidence of reducing first-use initialization cost, not a 
demonstrated steady-state speedup. The host was shared with another benchmark, 
and the round-to-round variation above limits broader performance claims.
   
   Across the 12 ordinary rounds and two diagnostic rounds, **920 benchmark 
queries completed with zero failures**. After every round, all 300 measurements 
returned by a `SELECT LAST *` sanity query matched exactly across versions, as 
did an additional seven-function query covering `max_time`, `last_value`, 
`first_value`, `max_by`, `min_by`, `extreme` and `mode`. These are targeted 
result checks rather than exhaustive dataset validation.
   
   ### Validation
   
   - The same patch on the historical benchmark revision passed the focused 
reactor tests (46 tests: 32 DataNode, 12 calc and 2 subscription; no 
failures/errors/skips) and a distribution package build.
   - On the latest master base (`4c34a253927`) plus this commit, 
`spotless:apply` and the focused reactor test command below completed 
successfully: **46 tests, zero failures/errors/skips** (32 DataNode, 12 calc 
and 2 subscription). `git diff --check` also passed.
   - Independent source review found no actionable issues; all six original 
coverage checks are retained in the unit test.
   
   Commands:
   
   ```sh
   mvn spotless:apply -o -pl iotdb-core/datanode -Ddevelocity.off=true
   mvn test -o -pl iotdb-core/datanode -am \
     -Dtest=TypeServicesTest,AccumulatorTest \
     -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false \
     -Ddevelocity.off=true
   ```
   
   This PR has:
   
   - [x] been self-reviewed.
   - [x] added a comment explaining the initialization cost.
   - [x] moved the existing coverage checks into a unit test.
   - [x] been tested with real queries against existing persisted aligned data 
in a 1 ConfigNode / 1 DataNode deployment.
   
   Key changed classes: `TypeServices.Aggregation`, `TypeServicesTest`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to