This is an automated email from the ASF dual-hosted git repository.
ethanfeng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 9a689b748 [CELEBORN-2028] Setup GA for grafana dashboard
9a689b748 is described below
commit 9a689b74825d6a009a603f677a151c60d68fc4ac
Author: Wang, Fei <[email protected]>
AuthorDate: Tue Jun 10 16:14:49 2025 +0800
[CELEBORN-2028] Setup GA for grafana dashboard
### What changes were proposed in this pull request?
Setup the GA for grafana dashboard.
1. Lint the dashboard with https://github.com/grafana/dashboard-linter
2. Check the duplicate id in dashboard json file
### Why are the changes needed?
It is helpful for grafana related PR review, for example:
https://github.com/apache/celeborn/pull/3307#discussion_r2134799722
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
GA.
<img width="1407" alt="image"
src="https://github.com/user-attachments/assets/35452633-ddff-4140-b929-3c44a943a2ab";
/>
Closes #3316 from turboFei/dashboard.
Authored-by: Wang, Fei <[email protected]>
Signed-off-by: mingji <[email protected]>
---
.github/workflows/grafana.yml | 53 +++++++++++++++++++++++++++++++
assets/grafana/.lint | 30 ++++++++++++++++++
dev/lint_grafana.py | 73 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 156 insertions(+)
diff --git a/.github/workflows/grafana.yml b/.github/workflows/grafana.yml
new file mode 100644
index 000000000..2632e213e
--- /dev/null
+++ b/.github/workflows/grafana.yml
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+name: Grafana Dashboard CI
+
+on:
+ push:
+ branches:
+ - main
+ - branch-*
+ pull_request:
+ branches:
+ - main
+ - branch-*
+
+jobs:
+ lint:
+ runs-on: ubuntu-latest
+ strategy:
+ fail-fast: false
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Go
+ uses: actions/setup-go@v4
+ with:
+ go-version: 'stable'
+ - uses: actions/setup-python@v5
+ with:
+ python-version: "3.12"
+ check-latest: true
+ # See https://github.com/grafana/dashboard-linter
+ - name: Install dashboard-linter
+ run: go install github.com/grafana/dashboard-linter@latest
+ - name: Lint Celeborn Dashboards
+ run: |
+ for dashboard in assets/grafana/*.json; do
+ python3 dev/lint_grafana.py "$dashboard"
+ dashboard-linter lint "$dashboard" --config assets/grafana/.lint
--strict
+ done
diff --git a/assets/grafana/.lint b/assets/grafana/.lint
new file mode 100644
index 000000000..6f86a8bd5
--- /dev/null
+++ b/assets/grafana/.lint
@@ -0,0 +1,30 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# See
https://github.com/grafana/dashboard-linter/blob/main/docs/index.md#exclusions-and-warnings
+
+exclusions:
+ template-datasource-rule:
+ reason: Celeborn uses ${DS_PROMETHEUS} variable in the template, which is
not supported by the rule.
+ panel-datasource-rule:
+ reason: Celeborn uses ${DS_PROMETHEUS} variable in the template, which is
not supported by the rule.
+ panel-title-description-rule:
+ reason: Some panels have a title but no description.
+ template-on-time-change-reload-rule:
+ reason: Allow dashboard variables not to reload on time change.
+ uneditable-dashboard:
+ reason: Celeborn dashboards are editable.
diff --git a/dev/lint_grafana.py b/dev/lint_grafana.py
new file mode 100755
index 000000000..55a21d97a
--- /dev/null
+++ b/dev/lint_grafana.py
@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import json
+import sys
+from collections import Counter
+
+def traverse_ids(obj, ids):
+ if isinstance(obj, dict):
+ for key, value in obj.items():
+ if key == "id":
+ ids.append(value)
+ traverse_ids(value, ids)
+ elif isinstance(obj, list):
+ for item in obj:
+ traverse_ids(item, ids)
+
+def check_ids(file_path):
+ with open(file_path, "r", encoding="utf-8") as f:
+ data = json.load(f)
+ ids = []
+ traverse_ids(data, ids)
+
+ # Find duplicates.
+ counts = Counter(ids)
+ duplicates = {k: v for k, v in counts.items() if v > 1}
+
+ # Collect numeric IDs.
+ numeric_ids = []
+ for id_value in ids:
+ try:
+ numeric_ids.append(int(id_value))
+ except (ValueError, TypeError):
+ continue
+
+ # print the current maximum numeric id.
+ if numeric_ids:
+ current_max = max(numeric_ids)
+ print(f"Current max numeric id: {current_max}")
+ else:
+ print("No numeric IDs found; cannot compute max numeric id.")
+
+ if duplicates:
+ print("Found duplicate id(s):")
+ for dup, count in duplicates.items():
+ print(f" {dup}: {count} times")
+ sys.exit(1)
+ else:
+ print("No duplicate id found.")
+
+if __name__ == "__main__":
+ if len(sys.argv) != 2:
+ print("Usage: python3 lint_grafana.py <path/to/dashboard.json>")
+ sys.exit(1)
+ file_path = sys.argv[1]
+ print(f"Linting: {file_path}")
+ check_ids(file_path)
