mingisme opened a new issue, #19331:
URL: https://github.com/apache/iotdb/issues/19331
### Search before asking
- [x] I searched in the issues and found nothing similar.
### Version
Reproduced on 2.0.8 (build `b88b5dc`) and on `master` at `a048ebf`
(2.0.13-SNAPSHOT).
### Describe the bug and provide the minimal reproduce step
Tree-model queries on devices that use a schema template fetch the schema
remotely on every run,
even right after the same query ran. Normal (non-template) devices are
served from the DataNode
schema cache after their first query. On a cluster with
`schema_replication_factor=3`, each
remote fetch is a Ratis read on the schema region. In our 3C3D deployment, a
40-device
`SELECT LAST` spent about 180 ms of its 575 ms average in schema fetch.
Steps on a standalone node:
```sql
CREATE DATABASE root.repro;
CREATE SCHEMA TEMPLATE t (m1 FLOAT, m2 FLOAT);
SET SCHEMA TEMPLATE t TO root.repro.tpl;
INSERT INTO root.repro.tpl.d1(timestamp, m1, m2) VALUES (1, 1.0, 2.0);
CREATE TIMESERIES root.repro.norm.m1 FLOAT;
INSERT INTO root.repro.norm(timestamp, m1) VALUES (1, 1.0);
-- Run each of these several times and read "Fetch Schema Cost".
EXPLAIN ANALYZE SELECT m1, m2 FROM root.repro.tpl.d1;
EXPLAIN ANALYZE SELECT m1 FROM root.repro.norm;
```
### What did you expect to see?
After the first run, `Fetch Schema Cost` for the template device is about
the same as for the
normal device, about 0.1 ms.
### What did you see instead?
On `master`, the template device costs 2 to 5 ms on every run. The normal
device costs 0.06 ms.
| Run | Template device | Normal device |
|---|---|---|
| 1 to 5 after warm-up | 4.808, 2.405, 2.261, 3.352, 2.056 ms | 0.075,
0.058, 0.066, 0.059, 0.061 ms |
### Anything else?
There are two causes. The second one shows only on a cluster, and only once
the first one is
fixed.
1. `ClusterSchemaFetcher.fetchSchema` calls
`schemaCache.getMatchedTemplateSchema(pattern.getDevicePath())`. Here
`pattern` is a
`PartialPath` for a full series path such as `root.db.d1.s1`, and
`PartialPath.getDevicePath()` returns the path itself. The cache stores
the device under
`root.db.d1`, so the lookup always misses.
2. On a cluster, a DataNode that runs the fragment of a `SELECT LAST`
coordinated by another
DataNode calls `declareLastCache`. That creates the device's cache entry
with a
`TreeDeviceNormalSchema`. Later, `TableDeviceCacheEntry.setDeviceSchema`
stores the fetched
template schema with `compareAndSet(null, ...)`, which can never replace
the normal schema. The
device then misses on that DataNode until restart.
On a local 1C3D cluster with 40 template devices, the 40-device `SELECT
LAST` costs 11.9 ms in
schema fetch on `master`. With fix 1 alone it still costs 12.0 ms, because
12 to 13 devices per
DataNode stay stuck. With both fixes it costs 0.37 ms.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]