shizy818 commented on PR #15277:
URL: https://github.com/apache/iotdb/pull/15277#issuecomment-2785505906
Performance
Case 1:
| Device 100，1M records for each device | distinct(device_id) |
count(distinct(device_id)) |
|--------------------------|-----------------------|------------------------------|
| master branch | 0.225s | 0.216s |
| distinct_tag branch | 0.038s | 0.040s
Case 2:
| Device 10000，1000 records for each device | distinct(device_id) |
count(distinct(device_id)) |
|--------------------------|-----------------------|------------------------------|
| master branch | 0.117s | 0.139s |
| distinct_tag branch | 0.145s | 0.123s
It works much better in case 1. There are about total 10 tsfile in each data
region. Each tsfile contains data from 100 devices. Now it reads one tsfile and
cease for distinct agg on tag column.
```
└── test_g_0
├── 25
│ ├── 2713
│ │ ├── 1744096954180-1-0-0.tsfile
│ │ └── 1744096954180-1-0-0.tsfile.resource
│ ├── 2714
│ │ ├── 1744096983125-1-0-0.tsfile
│ │ └── 1744096983125-1-0-0.tsfile.resource
│ ├── 2715
│ │ ├── 1744097017950-1-0-0.tsfile
│ │ └── 1744097017950-1-0-0.tsfile.resource
│ ├── 2716
│ │ ├── 1744097053553-1-0-0.tsfile
│ │ └── 1744097053553-1-0-0.tsfile.resource
│ ├── 2717
│ │ ├── 1744097088805-1-0-0.tsfile
│ │ └── 1744097088805-1-0-0.tsfile.resource
│ ├── 2718
│ │ ├── 1744097124269-1-0-0.tsfile
│ │ └── 1744097124269-1-0-0.tsfile.resource
│ ├── 2719
│ │ ├── 1744097159414-1-0-0.tsfile
│ │ └── 1744097159414-1-0-0.tsfile.resource
│ ├── 2720
│ │ └── 1744097195408-1-0-0.tsfile
│ └── 2721
│ └── 1744097231105-1-0-0.tsfile
└── 26
├── 2713
│ ├── 1744096954209-1-0-0.tsfile
│ └── 1744096954209-1-0-0.tsfile.resource
├── 2714
│ ├── 1744096983120-1-0-0.tsfile
│ └── 1744096983120-1-0-0.tsfile.resource
......
```
In case 2, devices scatter in different data regions. There's no obvious
improvement.
```
├── 10
│ └── 2713
│ ├── 1744095199739-1-0-0.tsfile
│ └── 1744095199739-1-0-0.tsfile.resource
├── 11
│ └── 2713
│ ├── 1744095200036-1-0-0.tsfile
│ └── 1744095200036-1-0-0.tsfile.resource
├── 4
│ └── 2713
│ ├── 1744095197967-1-0-0.tsfile
│ └── 1744095197967-1-0-0.tsfile.resource
├── 5
│ └── 2713
│ ├── 1744095198001-1-0-0.tsfile
│ └── 1744095198001-1-0-0.tsfile.resource
......
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
