jixuan1989 opened a new issue, #108:
URL: https://github.com/apache/iotdb-extras/issues/108
## Use Case
Consider a Grafana dashboard that monitors multiple IoT devices (or
services, products, etc.) across many metrics. Each metric is displayed in one
panel, and each panel shows one time-series curve per device.
For example: 8 devices × 20 metrics = 20 panels, each with 8 curves. You
want a **single multi-select dropdown variable** at the top of the dashboard so
that when users select which devices to display, **all 20 panels update
simultaneously** to show only the selected devices' curves.
### Data Model in IoTDB
The time series are modeled as `root.application.{device}.{metric}`:
```
root.application.device1.m1
root.application.device1.m2
...
root.application.device1.m20
root.application.device2.m1
root.application.device2.m2
...
root.application.device2.m20
...
root.application.device8.m1
root.application.device8.m2
...
root.application.device8.m20
```
### How Other Datasources Solve This
With SQL-based datasources (Prometheus, MySQL, PostgreSQL), this is trivial
— just inject the variable into a WHERE clause:
```sql
SELECT m1 FROM root.application WHERE device IN (${device})
```
With the Apache IoTDB plugin, the query model is `prefixPath[] +
expression[]`, and there's no WHERE clause. This makes the common "global
variable filter" pattern extremely difficult to implement.
## Current Behavior
In `src/datasource.ts`, `applyTemplateVariables` processes each prefixPath
element individually:
```typescript
if (query.prefixPath) {
query.prefixPath.map(
(_, index) => (query.prefixPath[index] =
getTemplateSrv().replace(query.prefixPath[index], scopedVars))
);
}
```
When a user writes:
```json
"prefixPath": ["root.application.${device}"]
```
And `${device}` is a multi-value variable with selected values `["device1",
"device2"]`, the result is:
- Default format: `root.application.device1,device2` — **invalid IoTDB path**
- Glob format: `root.application.{device1,device2}` — **also invalid**
The plugin does not expand a single prefixPath entry into multiple valid
paths.
## Expected Behavior
The plugin should detect multi-value variables in prefixPath and expand them:
```
Input: prefixPath = ["root.application.${device}"]
Selected values: ["device1", "device2"]
Output: prefixPath = ["root.application.device1", "root.application.device2"]
```
## Current Workaround (Complex)
Without this feature, to configure a single panel (e.g., metric `m1`), users
must:
**Step 1:** Hardcode all 8 device paths in prefixPath.
**Step 2:** Add a `filterFieldsByName` transformation that uses
`${device:pipe}` (Grafana's pipe formatter expands multi-value variables into
regex OR syntax like `device1|device2`) to filter the returned fields
client-side.
```json
{
"targets": [{
"refId": "A",
"sqlType": "SQL: Full Customized",
"expression": ["m1"],
"prefixPath": [
"root.application.device1",
"root.application.device2",
"root.application.device3",
"root.application.device4",
"root.application.device5",
"root.application.device6",
"root.application.device7",
"root.application.device8"
]
}],
"transformations": [{
"id": "filterFieldsByName",
"options": {
"include": {
"pattern": "time|.*(${device:pipe}).*"
}
}
}]
}
```
**This must be repeated for all 20 panels.**
### Problems with this workaround:
- **Verbose**: 20 panels × (8 hardcoded paths + 1 transformation) = 180
config items
- **Wasteful**: IoTDB always queries all 8 devices even when user only
selected 2 — unnecessary I/O and memory on the server
- **Fragile**: Forgetting the transformation on one panel means it silently
shows all devices unfiltered (hard to debug)
- **Maintenance burden**: Adding a 9th device requires editing all 20
panels' prefixPath arrays
- **Non-obvious**: The `time|.*(${device:pipe}).*` regex pattern is
confusing to new users
## With This Feature (Simple)
After the proposed change, the same panel configuration becomes:
```json
{
"targets": [{
"refId": "A",
"sqlType": "SQL: Full Customized",
"expression": ["m1"],
"prefixPath": ["root.application.${device}"]
}]
}
```
**That's it.** No transformations. No hardcoded paths. No post-filtering.
When user selects `device1` and `device2`, the plugin internally expands to:
```json
"prefixPath": ["root.application.device1", "root.application.device2"]
```
Only the selected devices are queried from IoTDB.
### Comparison
| Aspect | Current Workaround | With This Feature |
|--------|-------------------|-------------------|
| prefixPath per panel | N hardcoded paths (one per device) | 1 path with
`${variable}` |
| Transformation required | Yes, on every panel | No |
| Query efficiency | Always queries all N devices, filters client-side |
Only queries selected devices |
| Adding a new device | Update variable + edit all panels' prefixPath +
verify transformations | Update variable definition only |
| Config for 20 panels × 8 devices | 20 × (8 paths + 1 transformation) = 180
items | 20 × 1 path = 20 items |
| Debugging | Silent failures when transformation is missing | Standard
Grafana variable behavior |
## Proposed Implementation
The change is minimal (~15 lines in `src/datasource.ts`):
```typescript
applyTemplateVariables(query: IoTDBQuery, scopedVars: ScopedVars) {
const templateSrv = getTemplateSrv();
if (query.prefixPath) {
const expanded: string[] = [];
for (const path of query.prefixPath) {
if (templateSrv.containsTemplate(path)) {
// Use 'pipe' format to get individual values separated by '|'
const replaced = templateSrv.replace(path, scopedVars, 'pipe');
const values = replaced.split('|');
if (values.length > 1) {
// Multi-value variable detected: expand into multiple paths
for (const val of values) {
expanded.push(val);
}
} else {
expanded.push(replaced);
}
} else {
expanded.push(path);
}
}
query.prefixPath = expanded;
}
if (query.expression) {
query.expression = query.expression.map((e) => templateSrv.replace(e,
scopedVars));
}
if (query.condition) {
query.condition = templateSrv.replace(query.condition, scopedVars);
}
if (query.control) {
query.control = templateSrv.replace(query.control, scopedVars);
}
return query;
}
```
### Backward Compatibility
This change is fully backward-compatible and does not affect normal usage:
- **Paths without variables**: Literal paths like `root.application.device1`
contain no template syntax, so `containsTemplate()` returns false and they pass
through unchanged.
- **Single-value variables**: When `${device}` resolves to a single value
(e.g. `device1`), `replace()` returns `root.application.device1` with no `|`
separator, so `split('|')` produces a single-element array — same behavior as
today.
- **Non-prefixPath variables**: The `expression`, `condition`, and `control`
fields continue to use simple `replace()` as before.
Only when a prefixPath entry contains a multi-value variable with multiple
selected values does the new expansion logic activate.
## Environment
- Grafana: 11.4 (also tested on 9.3+)
- Plugin version: 1.0.1
- IoTDB: 1.3.x
## Summary
This is a small code change (~15 lines) with high user impact. The "global
variable to filter all panels" pattern is one of the most fundamental Grafana
dashboard patterns, and the current plugin makes it unnecessarily complex to
implement.
---
## 中文版说明
### 使用场景
假设有一个 Grafana 看板，同时监控多个 IoT
设备（或服务、产品等）的多项指标。每种指标用一个图表来展示，一个图表可以查看多个设备的数据（即有多少个设备，一个图表中就有多少条时间序列曲线）。
例如：8 个设备 × 20 个指标，则需要 20 个面板，每个面板有 8
条曲线。希望在看板顶部放一个下拉多选变量，让用户选择要查看哪些设备，则所有面板同步切换。
### IoTDB 中的数据建模
时间序列建模为 `root.application.{device}.{metric}`：
```
root.application.device1.m1
root.application.device1.m2
...
root.application.device1.m20
root.application.device2.m1
root.application.device2.m2
...
root.application.device2.m20
...
root.application.device8.m1
root.application.device8.m2
...
root.application.device8.m20
```
### 其他数据源如何解决
在 SQL 类数据源（Prometheus、MySQL、PostgreSQL）中，这很简单——在 WHERE 子句中注入变量即可：
```sql
SELECT m1 FROM root.application WHERE device IN (${device})
```
但在 Apache IoTDB 插件中，查询模型是 `prefixPath[] + expression[]`，没有 WHERE
子句可以注入变量。这使得"全局变量过滤"这一常见模式极难实现。
### 当前行为
插件的 `applyTemplateVariables`（`src/datasource.ts`）对 prefixPath 中的每个元素单独调用
`getTemplateSrv().replace()`。
当用户写：
```json
"prefixPath": ["root.application.${device}"]
```
且 `${device}` 是多选变量（选中 `device1` 和 `device2`）时，结果为：
- 默认格式：`root.application.device1,device2` — **无效的 IoTDB 路径**
- Glob 格式：`root.application.{device1,device2}` — **同样无效**
插件不会将一个 prefixPath 元素展开为多条有效路径。
### 期望行为
插件应检测 prefixPath 中的多选变量并展开：
```
输入：prefixPath = ["root.application.${device}"]，选中值 = ["device1", "device2"]
输出：prefixPath = ["root.application.device1", "root.application.device2"]
```
### 当前 Workaround（复杂）
没有此功能时，配置一个面板（如指标 `m1`）需要：
**第一步**：在 prefixPath 中硬编码所有 8 个设备路径。
**第二步**：添加 `filterFieldsByName` transformation，利用 `${device:pipe}`（Grafana 的
pipe 格式化器会将多选值展开为正则 OR 语法，如 `device1|device2`）在客户端过滤返回的字段。
```json
{
"targets": [{
"refId": "A",
"sqlType": "SQL: Full Customized",
"expression": ["m1"],
"prefixPath": [
"root.application.device1",
"root.application.device2",
"root.application.device3",
"root.application.device4",
"root.application.device5",
"root.application.device6",
"root.application.device7",
"root.application.device8"
]
}],
"transformations": [{
"id": "filterFieldsByName",
"options": {
"include": {
"pattern": "time|.*(${device:pipe}).*"
}
}
}]
}
```
**这必须对所有 20 个面板重复操作。**
### Workaround 的问题：
- **配置冗长**：20 面板 × (8 条硬编码路径 + 1 个 transformation) = 180 个配置项
- **资源浪费**：IoTDB 始终查询全部 8 个设备，即使用户只选了 2 个——不必要的 I/O 和内存开销
- **容易遗漏**：某个面板忘加 transformation 就会静默显示所有设备数据（难以调试）
- **维护负担**：新增第 9 个设备需要修改全部 20 个面板的 prefixPath 数组
- **不直观**：`time|.*(${device:pipe}).*` 这个正则对新用户来说很难理解
### 改造后（简单）
改造后，同一个面板的配置变为：
```json
{
"targets": [{
"refId": "A",
"sqlType": "SQL: Full Customized",
"expression": ["m1"],
"prefixPath": ["root.application.${device}"]
}]
}
```
**就这样。** 无需 transformation，无需硬编码路径，无需客户端后过滤。
当用户选择 `device1` 和 `device2` 时，插件内部展开为：
```json
"prefixPath": ["root.application.device1", "root.application.device2"]
```
只查询选中的设备。
### 改进前后对比
| 维度 | 当前（Workaround） | 改进后 |
|------|-------------------|--------|
| 每面板 prefixPath | N 条硬编码路径（每设备一条） | 1 条含 `${variable}` 的模板路径 |
| 是否需要 Transformation | 每个面板都要加 | 不需要 |
| 查询效率 | 始终查全部 N 个设备，客户端过滤 | 只查选中的设备 |
| 新增设备 | 改变量 + 改所有面板 prefixPath + 验证 transformation | 只改变量定义 |
| 20 面板 × 8 设备的配置量 | 20 × (8 路径 + 1 transformation) = 180 项 | 20 × 1 路径 = 20
项 |
| 调试难度 | 遗漏 transformation 时静默失败 | 标准 Grafana 变量行为 |
### 向后兼容性
此改动完全向后兼容，不影响正常使用：
- **不含变量的路径**：字面量路径如 `root.application.device1` 不包含模板语法，`containsTemplate()`
返回 false，直接原样传递，行为不变。
- **单值变量**：当 `${device}` 只选中一个值（如 `device1`）时，`replace()` 返回
`root.application.device1`，不含 `|` 分隔符，`split('|')` 产生单元素数组——与当前行为完全一致。
- **非 prefixPath 字段**：`expression`、`condition`、`control` 字段继续使用简单的
`replace()`，不受影响。
只有当 prefixPath 元素包含多选变量且选中了多个值时，新的展开逻辑才会激活。
### 建议实现
改动量约 15 行代码（`src/datasource.ts`），完全向后兼容。详见上方英文版的 Proposed Implementation 部分。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
