mingisme opened a new issue, #19331:
URL: https://github.com/apache/iotdb/issues/19331

   ### Search before asking
   
   - [x] I searched in the issues and found nothing similar.
   
   ### Version
   
   Reproduced on 2.0.8 (build `b88b5dc`) and on `master` at `a048ebf` 
(2.0.13-SNAPSHOT).
   
   ### Describe the bug and provide the minimal reproduce step
   
   Tree-model queries on devices that use a schema template fetch the schema 
remotely on every run,
   even right after the same query ran. Normal (non-template) devices are 
served from the DataNode
   schema cache after their first query. On a cluster with 
`schema_replication_factor=3`, each
   remote fetch is a Ratis read on the schema region. In our 3C3D deployment, a 
40-device
   `SELECT LAST` spent about 180 ms of its 575 ms average in schema fetch.
   
   Steps on a standalone node:
   
   ```sql
   CREATE DATABASE root.repro;
   CREATE SCHEMA TEMPLATE t (m1 FLOAT, m2 FLOAT);
   SET SCHEMA TEMPLATE t TO root.repro.tpl;
   INSERT INTO root.repro.tpl.d1(timestamp, m1, m2) VALUES (1, 1.0, 2.0);
   CREATE TIMESERIES root.repro.norm.m1 FLOAT;
   INSERT INTO root.repro.norm(timestamp, m1) VALUES (1, 1.0);
   
   -- Run each of these several times and read "Fetch Schema Cost".
   EXPLAIN ANALYZE SELECT m1, m2 FROM root.repro.tpl.d1;
   EXPLAIN ANALYZE SELECT m1 FROM root.repro.norm;
   ```
   
   ### What did you expect to see?
   
   After the first run, `Fetch Schema Cost` for the template device is about 
the same as for the
   normal device, about 0.1 ms.
   
   ### What did you see instead?
   
   On `master`, the template device costs 2 to 5 ms on every run. The normal 
device costs 0.06 ms.
   
   | Run | Template device | Normal device |
   |---|---|---|
   | 1 to 5 after warm-up | 4.808, 2.405, 2.261, 3.352, 2.056 ms | 0.075, 
0.058, 0.066, 0.059, 0.061 ms |
   
   ### Anything else?
   
   There are two causes. The second one shows only on a cluster, and only once 
the first one is
   fixed.
   
   1. `ClusterSchemaFetcher.fetchSchema` calls
      `schemaCache.getMatchedTemplateSchema(pattern.getDevicePath())`. Here 
`pattern` is a
      `PartialPath` for a full series path such as `root.db.d1.s1`, and
      `PartialPath.getDevicePath()` returns the path itself. The cache stores 
the device under
      `root.db.d1`, so the lookup always misses.
   2. On a cluster, a DataNode that runs the fragment of a `SELECT LAST` 
coordinated by another
      DataNode calls `declareLastCache`. That creates the device's cache entry 
with a
      `TreeDeviceNormalSchema`. Later, `TableDeviceCacheEntry.setDeviceSchema` 
stores the fetched
      template schema with `compareAndSet(null, ...)`, which can never replace 
the normal schema. The
      device then misses on that DataNode until restart.
   
   On a local 1C3D cluster with 40 template devices, the 40-device `SELECT 
LAST` costs 11.9 ms in
   schema fetch on `master`. With fix 1 alone it still costs 12.0 ms, because 
12 to 13 devices per
   DataNode stay stuck. With both fixes it costs 0.37 ms.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to